Oct 30 00:35:49.519355 ip-10-0-131-254 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Oct 30 00:35:49.519367 ip-10-0-131-254 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Oct 30 00:35:49.519374 ip-10-0-131-254 systemd[1]: kubelet.service: Failed with result 'resources'.
Oct 30 00:35:49.519608 ip-10-0-131-254 systemd[1]: Failed to start Kubernetes Kubelet.
Oct 30 00:35:59.757993 ip-10-0-131-254 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Oct 30 00:35:59.758008 ip-10-0-131-254 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7b8c16a63c5445c898f571ef7377dd82 --
Oct 30 00:38:05.522471 ip-10-0-131-254 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 00:38:05.887199 ip-10-0-131-254 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:38:05.887199 ip-10-0-131-254 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 00:38:05.887199 ip-10-0-131-254 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:38:05.887199 ip-10-0-131-254 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:38:05.891981 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.891886    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.891981 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.891977    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.891981 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.891985    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.891981 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.891990    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.891995    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892000    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892006    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892011    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892015    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892019    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892023    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892027    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892031    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892043    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892048    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892051    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892056    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892059    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892064    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892068    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892072    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892076    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.892265 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892080    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892083    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892087    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892090    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892094    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892098    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892102    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892106    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892110    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892114    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892118    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892122    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892126    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892131    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892135    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892140    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892145    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892150    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892154    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892158    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.893079 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892162    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892167    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892189    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892194    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892200    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892205    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892209    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892213    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892217    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892221    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892225    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892229    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892234    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892238    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892243    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892247    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892251    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892256    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892260    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892264    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.893924 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892268    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892271    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892277    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892281    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892285    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892289    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892294    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892299    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892314    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892319    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892323    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892328    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892332    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892336    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892341    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892345    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892349    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892354    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892359    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892363    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.895688 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892368    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892372    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892376    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892381    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892386    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892390    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892394    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892398    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892402    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892407    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892411    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892415    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892420    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892425    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892430    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892434    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892441    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892445    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892449    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.896164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892454    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892459    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892464    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892468    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.892472    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893927    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893937    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893942    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893946    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893950    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893956    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893966    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893971    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893975    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893980    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893984    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893988    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893993    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.893997    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.896665 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894001    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894007    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894014    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894019    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894032    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894038    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894043    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894049    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894054    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894059    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894063    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894068    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894072    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894077    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894081    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894086    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894090    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894095    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894099    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894104    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.897168 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894108    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894112    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894116    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894120    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894124    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894140    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894144    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894148    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894152    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894157    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894161    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894165    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894169    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894193    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894198    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894204    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894208    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894213    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894216    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.898064 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894222    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894226    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894229    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894231    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894234    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894237    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894240    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894243    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894245    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894248    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894250    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894253    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894256    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894258    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894261    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894263    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894266    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894268    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894271    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894280    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.898900 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894283    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894286    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894288    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894291    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894293    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894296    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894299    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894301    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894304    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894306    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894309    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894311    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894315    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894317    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894320    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894322    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894325    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894327    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894332    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894335    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.899596 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894338    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894340    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894343    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894345    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894348    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894350    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894353    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894355    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894360    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894364    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894368    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894371    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894374    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894915    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894920    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894923    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894925    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894928    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894933    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.900366 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894937    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894940    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894943    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894946    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894949    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894953    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894956    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894959    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894961    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894964    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894967    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894989    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894993    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.894997    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895000    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895004    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895008    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895012    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895016    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895020    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.900870 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895024    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895028    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895032    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895035    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895039    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895044    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895048    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895052    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895056    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895060    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895064    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895068    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895072    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895077    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895082    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895087    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895091    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895096    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895100    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.901371 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895104    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895109    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895114    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895118    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895125    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895129    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895133    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895138    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895143    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895147    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895151    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895155    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895160    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895164    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895168    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895191    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895196    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895200    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895204    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895209    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.901842 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895214    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895219    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895225    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895229    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895233    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895237    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895242    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895246    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895251    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895255    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895259    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895264    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895268    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895273    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895277    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895281    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895289    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895293    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895297    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.902355 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895302    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895306    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895310    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895314    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895318    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895322    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895326    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895331    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895335    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895339    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895344    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895348    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895352    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895357    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895361    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895366    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895370    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895374    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895378    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895382    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.902808 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895386    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.895392    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897071    2571 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897088    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897104    2571 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897112    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897119    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897124    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897131    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897139    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897144    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897150    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897155    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897161    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897165    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897187    2571 flags.go:64] FLAG: --cgroup-root=""
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897192    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897197    2571 flags.go:64] FLAG: --client-ca-file=""
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897201    2571 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897206    2571 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897215    2571 flags.go:64] FLAG: --cluster-domain=""
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897220    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897226    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897231    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 00:38:05.903289 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897236    2571 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897246    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897253    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897258    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897264    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897269    2571 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897274    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897279    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897284    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897289    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897296    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897301    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897306    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897311    2571 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897316    2571 flags.go:64] FLAG: --enable-server="true"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897321    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897327    2571 flags.go:64] FLAG: --event-burst="100"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897333    2571 flags.go:64] FLAG: --event-qps="50"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897338    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897343    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897348    2571 flags.go:64] FLAG: --eviction-hard=""
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897355    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897361    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897366    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897371    2571 flags.go:64] FLAG: --eviction-soft=""
Oct 30 00:38:05.903955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897376    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897381    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897386    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897390    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897395    2571 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897400    2571 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897405    2571 flags.go:64] FLAG: --feature-gates=""
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897415    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897422    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897428    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897434    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897439    2571 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897444    2571 flags.go:64] FLAG: --help="false"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897449    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-131-254.ec2.internal"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897455    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897459    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897464    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897470    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897476    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897481    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897486    2571 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897491    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897495    2571 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 00:38:05.904640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897500    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897506    2571 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897510    2571 flags.go:64] FLAG: --kube-reserved=""
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897515    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897520    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897526    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897530    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897535    2571 flags.go:64] FLAG: --lock-file=""
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897540    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897545    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897549    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897560    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897565    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897570    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897575    2571 flags.go:64] FLAG: --logging-format="text"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897579    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897585    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897592    2571 flags.go:64] FLAG: --manifest-url=""
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897597    2571 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897605    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897610    2571 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897617    2571 flags.go:64] FLAG: --max-pods="110"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897622    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897627    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897632    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 00:38:05.905259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897637    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897642    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897647    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897652    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897663    2571 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897668    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897673    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897678    2571 flags.go:64] FLAG: --pod-cidr=""
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897683    2571 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897688    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897693    2571 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897697    2571 flags.go:64] FLAG: --port="10250"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897702    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897707    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09906a28b26f72a39"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897712    2571 flags.go:64] FLAG: --qos-reserved=""
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897717    2571 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897722    2571 flags.go:64] FLAG: --register-node="true"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897727    2571 flags.go:64] FLAG: --register-with-taints=""
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897734    2571 flags.go:64] FLAG: --registry-burst="10"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897738    2571 flags.go:64] FLAG: --registry-qps="5"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897743    2571 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897747    2571 flags.go:64] FLAG: --reserved-memory=""
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897754    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897758    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897765    2571 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897770    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 00:38:05.905861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897775    2571 flags.go:64] FLAG: --runonce="false"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897780    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897786    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897793    2571 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897798    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897813    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897819    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897824    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897830    2571 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897834    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897839    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897844    2571 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897849    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897853    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897858    2571 flags.go:64] FLAG: --system-cgroups=""
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897863    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897872    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897877    2571 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897882    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897889    2571 flags.go:64] FLAG: --tls-min-version=""
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897893    2571 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897898    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897903    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897908    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897914    2571 flags.go:64] FLAG: --v="2"
Oct 30 00:38:05.906514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897921    2571 flags.go:64] FLAG: --version="false"
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897927    2571 flags.go:64] FLAG: --vmodule=""
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897934    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.897939    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898229    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898240    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898245    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898250    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898255    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898260    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898266    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898271    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898275    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898279    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898283    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898287    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898291    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898296    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898300    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898304    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898308    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.907099 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898312    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898317    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898321    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898325    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898329    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898333    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898338    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898342    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898346    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898351    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898355    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898359    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898363    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898367    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898371    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898375    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898379    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898384    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898388    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898393    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.907693 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898397    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898403    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898412    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898419    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898425    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898430    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898434    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898438    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898442    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898447    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898451    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898455    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898462    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898469    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898474    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898479    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898483    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898488    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898492    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.908164 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898496    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898500    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898505    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898509    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898513    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898517    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898521    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898526    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898530    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898534    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898538    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898542    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898547    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898551    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898557    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898561    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898566    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898570    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898574    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.908673 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898578    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898583    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898587    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898591    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898595    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898600    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898604    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898608    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898612    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898616    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898620    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898626    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898630    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898634    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898639    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898643    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898647    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898651    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898656    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.909121 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898660    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898664    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898669    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898674    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898678    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898682    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898686    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898691    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898696    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898700    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898705    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898710    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.898753    2571 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898811    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898818    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898823    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.909586 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898827    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898832    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898836    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898840    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898844    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898848    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898852    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898856    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898861    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898866    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898870    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898874    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898879    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898883    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898887    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898894    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898898    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898902    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898906    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.910011 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898911    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898915    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898919    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898923    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898928    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898934    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898939    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898943    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898948    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898953    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898957    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898962    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898966    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898970    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898974    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898979    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898983    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898987    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898991    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.910496 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.898995    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899001    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899005    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899009    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899014    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899019    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899025    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899029    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899033    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899040    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899044    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899048    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899053    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899057    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899061    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899065    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899069    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899073    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899079    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899084    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.910986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899088    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899092    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899096    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899101    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899106    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899110    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899115    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899119    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899123    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899127    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899131    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899136    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899140    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899144    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899192    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899198    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899203    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899206    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899210    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899213    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.911554 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899217    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899222    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899226    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899229    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899232    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899236    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899238    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899241    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899244    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899246    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899251    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899258    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899262    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899265    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899268    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899270    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899273    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899276    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899278    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.912049 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899284    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899286    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899290    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899293    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899296    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.899299    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.906431    2571 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.906448    2571 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.906465    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.906470    2571 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906529    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906535    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906538    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906542    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906547    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906551    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906554    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906556    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906559    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.912609 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906562    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906564    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906567    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906570    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906572    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906575    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906578    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906580    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906583    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906585    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906589    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906591    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906595    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906598    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906600    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906603    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906606    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906608    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906611    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.913298 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906613    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906617    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906620    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906622    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906625    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906627    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906630    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906632    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906635    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906638    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906640    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906643    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906646    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906648    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906651    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906653    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906655    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906658    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906661    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906663    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.913986 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906666    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906668    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906671    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906673    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906676    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906679    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906683    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906686    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906689    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906692    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906695    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906697    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906700    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906703    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906706    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906709    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906711    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906714    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906717    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.914505 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906719    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906722    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906724    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906727    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906729    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906731    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906734    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906737    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906739    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906742    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906744    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906747    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906754    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906757    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906760    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906763    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906765    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906768    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906771    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906773    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.915357 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906776    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906778    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906781    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906783    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906786    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906788    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906791    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906793    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906796    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906798    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906801    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906803    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906805    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906808    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906810    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906813    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906815    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906817    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.915828 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.906820    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.906847    2571 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907004    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907008    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907011    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907014    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907016    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907020    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907022    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907025    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907028    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907030    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907033    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907035    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907038    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.916292 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907041    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907043    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907046    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907048    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907051    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907054    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907056    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907058    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907061    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907063    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907066    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907068    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907072    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907076    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907079    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907083    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907086    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907089    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907091    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.916677 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907094    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907096    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907098    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907101    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907103    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907106    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907110    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907112    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907115    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907117    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907120    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907123    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907125    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907127    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907130    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907132    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907135    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907137    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907140    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907143    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.917269 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907145    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907148    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907150    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907153    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907155    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907158    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907160    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907162    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907165    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907167    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907169    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907190    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907194    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907198    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907201    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907203    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907206    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907208    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907211    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907214    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.917769 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907216    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907219    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907221    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907224    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907227    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907229    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907232    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907234    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907237    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907240    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907243    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907245    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907248    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907250    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907253    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907255    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907258    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907260    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907263    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907265    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.918274 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907268    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907270    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907273    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907275    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907277    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907280    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907282    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907285    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907287    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907290    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907292    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907295    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907298    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907300    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.907323    2571 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907356    2571 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:38:05.918767 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907361    2571 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907364    2571 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907367    2571 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907370    2571 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907373    2571 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907376    2571 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907378    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907381    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907383    2571 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907386    2571 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907388    2571 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907391    2571 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907393    2571 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907396    2571 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907398    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907401    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907403    2571 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907406    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907408    2571 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907410    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:38:05.919145 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907413    2571 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907415    2571 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907418    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907420    2571 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907423    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907425    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907427    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907430    2571 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907432    2571 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907435    2571 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907438    2571 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907440    2571 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907443    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907445    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907448    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907451    2571 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907453    2571 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907456    2571 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907458    2571 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907461    2571 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:38:05.919638 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907463    2571 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907465    2571 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907468    2571 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907470    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907472    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907475    2571 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907477    2571 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907480    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907482    2571 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907485    2571 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907487    2571 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907490    2571 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907492    2571 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907495    2571 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907497    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907500    2571 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907502    2571 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907505    2571 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907507    2571 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907512    2571 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:38:05.920119 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907515    2571 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907518    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907521    2571 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907523    2571 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907526    2571 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907528    2571 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907531    2571 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907534    2571 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907536    2571 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907538    2571 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907541    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907543    2571 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907546    2571 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907548    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907551    2571 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907553    2571 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907556    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907558    2571 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907561    2571 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:38:05.920603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907563    2571 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907565    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907568    2571 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907570    2571 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907572    2571 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907575    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907577    2571 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907583    2571 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907587    2571 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907591    2571 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907594    2571 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907599    2571 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907602    2571 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907605    2571 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907607    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907609    2571 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907612    2571 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907615    2571 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:38:05.921060 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907617    2571 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907620    2571 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907623    2571 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907625    2571 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907628    2571 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907630    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907633    2571 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:05.907635    2571 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.907694    2571 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.909715    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.910583    2571 server.go:1014] "Starting client certificate rotation"
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.910683    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.911421    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.912461    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:38:05.921517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.913594    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:38:05.924153 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.924137    2571 log.go:25] "Validated CRI v1 runtime API"
Oct 30 00:38:05.927758 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.927744    2571 log.go:25] "Validated CRI v1 image API"
Oct 30 00:38:05.928764 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.928740    2571 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 00:38:05.932940 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.932921    2571 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ab451a2b-7a30-4408-bec0-cb085eaaf3c4:/dev/nvme0n1p3 cc0a2405-f468-4b26-9bea-30b0b7254fce:/dev/nvme0n1p4]
Oct 30 00:38:05.932987 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.932940    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 00:38:05.939466 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.939343    2571 manager.go:217] Machine: {Timestamp:2026-10-30 00:38:05.938026802 +0000 UTC m=+0.302834417 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102987 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec298aa3bcb77a693f417c968382affd SystemUUID:ec298aa3-bcb7-7a69-3f41-7c968382affd BootID:7b8c16a6-3c54-45c8-98f5-71ef7377dd82 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:f5:4f:5f:19 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:f5:4f:5f:19 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:b4:cc:d8:04:73 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 00:38:05.939466 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.939456    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 00:38:05.939602 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.939543    2571 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 00:38:05.942107 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.942076    2571 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 00:38:05.942257 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.942108    2571 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-254.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 00:38:05.942307 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.942259    2571 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 00:38:05.942307 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.942266    2571 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 00:38:05.942365 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.942307    2571 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:38:05.942365 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.942326    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:38:05.943015 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.943006    2571 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 00:38:05.945547 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.945535    2571 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 00:38:05.945661 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.945651    2571 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 00:38:05.946296 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.946287    2571 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 00:38:05.946328 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.946299    2571 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 00:38:05.946328 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.946310    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 00:38:05.946328 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.946318    2571 kubelet.go:405] "Adding apiserver pod source"
Oct 30 00:38:05.946328 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.946326    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 00:38:05.946644 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.946632    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:38:05.948668 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.948644    2571 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 00:38:05.953734 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.953720    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:38:05.954700 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954686    2571 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 00:38:05.954746 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954725    2571 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 00:38:05.954880 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954871    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 00:38:05.954910 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954886    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 00:38:05.954910 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954892    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 00:38:05.954910 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954899    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 00:38:05.954910 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954904    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 00:38:05.954910 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.954910    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 00:38:05.955276 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955266    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 00:38:05.955306 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955277    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 00:38:05.955306 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955284    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 00:38:05.955306 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955290    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 00:38:05.955306 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955298    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 00:38:05.955306 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955307    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 00:38:05.955817 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955807    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 00:38:05.955845 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.955821    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 00:38:05.959640 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.959625    2571 server.go:1263] "Started kubelet"
Oct 30 00:38:05.959722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.959674    2571 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 00:38:05.960149 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.960103    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 00:38:05.960221 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.960168    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 00:38:05.960474 ip-10-0-131-254 systemd[1]: Started Kubernetes Kubelet.
Oct 30 00:38:05.961837 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.961813    2571 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 00:38:05.963133 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.963116    2571 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 00:38:05.966947 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:05.966928    2571 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 00:38:05.967920 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.967898    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 00:38:05.968250 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.968236    2571 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 00:38:05.968341 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:05.968325    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:05.968880 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.968865    2571 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 00:38:05.968984 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.968973    2571 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 00:38:05.969114 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.968869    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 00:38:05.969209 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969112    2571 factory.go:55] Registering systemd factory
Oct 30 00:38:05.969209 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969130    2571 factory.go:223] Registration of the systemd container factory successfully
Oct 30 00:38:05.969209 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969169    2571 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 00:38:05.969209 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969201    2571 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 00:38:05.970298 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969383    2571 factory.go:153] Registering CRI-O factory
Oct 30 00:38:05.970298 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969416    2571 factory.go:223] Registration of the crio container factory successfully
Oct 30 00:38:05.970298 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969476    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 00:38:05.970298 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969500    2571 factory.go:103] Registering Raw factory
Oct 30 00:38:05.970298 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.969512    2571 manager.go:1199] Started watching for new ooms in manager
Oct 30 00:38:05.970669 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.970450    2571 manager.go:319] Starting recovery of all containers
Oct 30 00:38:05.985142 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.985125    2571 manager.go:324] Recovery completed
Oct 30 00:38:05.990151 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.990135    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:38:05.992060 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.992044    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:38:05.992118 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.992079    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:38:05.992118 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.992092    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:38:05.992591 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.992579    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:38:05.992657 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.992581    2571 cpu_manager.go:226] "Starting" policy="none"
Oct 30 00:38:05.992657 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.992611    2571 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 00:38:05.992657 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.992629    2571 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 00:38:05.993234 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.993219    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:38:05.993282 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.993254    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:38:05.993282 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.993267    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:38:05.993715 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.993704    2571 policy_none.go:50] "Start"
Oct 30 00:38:05.993747 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.993719    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 00:38:05.993747 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.993729    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 00:38:05.994847 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.994836    2571 policy_none.go:44] "Start"
Oct 30 00:38:05.998502 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:05.998478    2571 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-254.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 00:38:05.998502 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:05.998487    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-254.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 00:38:05.999314 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:05.998472    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-254.ec2.internal.18e327a996f907ae  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-254.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-254.ec2.internal,},FirstTimestamp:2026-10-30 00:38:05.959448494 +0000 UTC m=+0.324256110,LastTimestamp:2026-10-30 00:38:05.959448494 +0000 UTC m=+0.324256110,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-254.ec2.internal,}"
Oct 30 00:38:06.003997 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.003976    2571 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 00:38:06.015793 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.015696    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-254.ec2.internal.18e327a998eaaa41  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-254.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-131-254.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-131-254.ec2.internal,},FirstTimestamp:2026-10-30 00:38:05.992061505 +0000 UTC m=+0.356869124,LastTimestamp:2026-10-30 00:38:05.992061505 +0000 UTC m=+0.356869124,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-254.ec2.internal,}"
Oct 30 00:38:06.016300 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.016280    2571 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Oct 30 00:38:06.016300 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.016281    2571 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-254.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Oct 30 00:38:06.016393 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.016281    2571 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver"
Oct 30 00:38:06.022156 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.022136    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nv22s"
Oct 30 00:38:06.027550 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.027534    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-nv22s"
Oct 30 00:38:06.032932 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.032905    2571 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 00:38:06.035163 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.035153    2571 manager.go:341] "Starting Device Plugin manager"
Oct 30 00:38:06.035239 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.035195    2571 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 00:38:06.035239 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.035206    2571 server.go:87] "Starting device plugin registration server"
Oct 30 00:38:06.035491 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.035478    2571 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 00:38:06.035569 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.035493    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 00:38:06.035616 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.035577    2571 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 00:38:06.035668 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.035654    2571 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 00:38:06.035668 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.035660    2571 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 00:38:06.036392 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.036374    2571 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 00:38:06.036478 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.036426    2571 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.038792 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.038697    2571 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-254.ec2.internal.18e327a998eb0a32  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-254.ec2.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-131-254.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-131-254.ec2.internal,},FirstTimestamp:2026-10-30 00:38:05.992086066 +0000 UTC m=+0.356893681,LastTimestamp:2026-10-30 00:38:05.992086066 +0000 UTC m=+0.356893681,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-254.ec2.internal,}"
Oct 30 00:38:06.091345 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.091314    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 00:38:06.092665 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.092648    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 00:38:06.092665 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.092667    2571 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 00:38:06.092801 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.092688    2571 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 00:38:06.092801 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.092729    2571 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 00:38:06.110805 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.110785    2571 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:38:06.136491 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.136467    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:38:06.137455 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.137414    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:38:06.137455 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.137445    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:38:06.137455 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.137454    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:38:06.137573 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.137483    2571 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.155134 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.155114    2571 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.155201 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.155142    2571 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-254.ec2.internal\": node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.192897 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.192846    2571 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"]
Oct 30 00:38:06.193055 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.192968    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:38:06.193907 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.193889    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:38:06.193986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.193921    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:38:06.193986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.193931    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:38:06.195159 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.195146    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:38:06.195322 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.195308    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.195362 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.195339    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:38:06.195918 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.195900    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:38:06.195987 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.195936    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:38:06.195987 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.195947    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:38:06.195987 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.195908    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:38:06.196073 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.196006    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:38:06.196073 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.196017    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:38:06.197357 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.197341    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.197440 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.197366    2571 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:38:06.197882 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.197862    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.198155 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.198141    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:38:06.198246 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.198166    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:38:06.198246 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.198202    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:38:06.224942 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.224918    2571 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-254.ec2.internal\" not found" node="ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.228434 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.228414    2571 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-254.ec2.internal\" not found" node="ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.270287 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.270251    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.270287 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.270290    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.270483 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.270311    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a3c183736bfb0e3d8475eab25325f35-config\") pod \"kube-apiserver-proxy-ip-10-0-131-254.ec2.internal\" (UID: \"7a3c183736bfb0e3d8475eab25325f35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.298393 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.298369    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.370552 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.370518    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.370552 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.370550    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.370742 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.370569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a3c183736bfb0e3d8475eab25325f35-config\") pod \"kube-apiserver-proxy-ip-10-0-131-254.ec2.internal\" (UID: \"7a3c183736bfb0e3d8475eab25325f35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.370742 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.370629    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.370742 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.370642    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.370742 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.370629    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a3c183736bfb0e3d8475eab25325f35-config\") pod \"kube-apiserver-proxy-ip-10-0-131-254.ec2.internal\" (UID: \"7a3c183736bfb0e3d8475eab25325f35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.399491 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.399420    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.500099 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.499923    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.527783 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.527748    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.531468 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.531230    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:06.600922 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.600878    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.701457 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.701374    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.802015 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.801990    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.902594 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:06.902557    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:06.910733 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.910705    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 00:38:06.910868 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:06.910849    2571 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 00:38:07.003711 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:07.003647    2571 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:38:07.021504 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:07.021468    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e516b0f6958377721da4db26ec47f8e.slice/crio-4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc WatchSource:0}: Error finding container 4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc: Status 404 returned error can't find the container with id 4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc
Oct 30 00:38:07.021906 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:07.021881    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3c183736bfb0e3d8475eab25325f35.slice/crio-e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3 WatchSource:0}: Error finding container e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3: Status 404 returned error can't find the container with id e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3
Oct 30 00:38:07.027411 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.027395    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:38:07.030611 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.030452    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 00:33:06 +0000 UTC" deadline="2028-05-25 15:08:03.693566465 +0000 UTC"
Oct 30 00:38:07.030611 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.030472    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13766h29m56.663097203s"
Oct 30 00:38:07.046188 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.046159    2571 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:38:07.068702 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.068681    2571 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:07.090656 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.090635    2571 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:38:07.092371 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.092355    2571 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:38:07.097040 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.096993    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal" event={"ID":"7a3c183736bfb0e3d8475eab25325f35","Type":"ContainerStarted","Data":"e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3"}
Oct 30 00:38:07.098080 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.098060    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal" event={"ID":"3e516b0f6958377721da4db26ec47f8e","Type":"ContainerStarted","Data":"4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc"}
Oct 30 00:38:07.110009 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.109989    2571 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:38:07.626955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.626919    2571 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:38:07.651932 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.651905    2571 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:38:07.806157 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.806123    2571 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:38:07.947875 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.947837    2571 apiserver.go:52] "Watching apiserver"
Oct 30 00:38:07.968953 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.968930    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 00:38:07.977858 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.977831    2571 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 00:38:07.978226 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.978185    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-p8fxn","openshift-image-registry/node-ca-g6srk","openshift-multus/multus-ddf5d","openshift-network-operator/iptables-alerter-m6vlx","openshift-ovn-kubernetes/ovnkube-node-vwlnt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2","openshift-dns/node-resolver-ql49r","openshift-multus/network-metrics-daemon-52dwv","kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal","openshift-multus/multus-additional-cni-plugins-ckp4t","kube-system/global-pull-secret-syncer-x2s8q","openshift-cluster-node-tuning-operator/tuned-5sv2w","openshift-network-diagnostics/network-check-target-pgl9q"]
Oct 30 00:38:07.979786 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.979762    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:07.981383 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.981362    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:07.982377 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.982355    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:07.983467 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.983446    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:07.985706 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.985688    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:07.987577 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.987558    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:07.988388 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.988372    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:07.988480 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:07.988447    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:07.990318 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.990296    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:07.992232 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.991640    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:07.994049 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.993911    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:07.994049 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:07.993998    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:07.996195 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.996164    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 00:38:07.996600 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.996580    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 00:38:07.996731 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.996581    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q2s6t\""
Oct 30 00:38:07.997357 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.997335    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 00:38:07.997459 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.997443    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kg5gj\""
Oct 30 00:38:07.998889 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.998857    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:38:07.998991 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.998913    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-tmln4\""
Oct 30 00:38:07.998991 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.998959    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 00:38:07.998991 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.998975    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 00:38:07.998991 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.998976    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 00:38:07.999229 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.999100    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 00:38:07.999229 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.999114    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5s9q7\""
Oct 30 00:38:07.999229 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.999184    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 00:38:07.999229 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.999210    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 00:38:07.999440 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.999301    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:07.999831 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:07.999813    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:38:08.000197 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.000153    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.000305 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.000235    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:08.000648 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.000624    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 00:38:08.000648 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.000627    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 00:38:08.000959 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.000939    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ckcc7\""
Oct 30 00:38:08.001259 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.001240    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 00:38:08.001398 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.001281    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 00:38:08.001492 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.001309    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 00:38:08.001492 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.001312    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 00:38:08.002577 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.002556    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 00:38:08.002666 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.002613    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 00:38:08.002728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.002687    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t5wsq\""
Oct 30 00:38:08.002728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.002712    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 00:38:08.005642 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.005621    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 00:38:08.007272 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.007237    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-545pj\""
Oct 30 00:38:08.007402 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.007368    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 00:38:08.007402 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.007238    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 00:38:08.022873 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.022852    2571 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 00:38:08.070548 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.070523    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 00:38:08.077735 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077712    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-node-log\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.077890 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077748    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zfkx\" (UniqueName: \"kubernetes.io/projected/01a7c97e-b248-4348-a07a-77652f9ce3c9-kube-api-access-8zfkx\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.077890 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077773    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-serviceca\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.077890 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-kubelet-config\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.077890 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077813    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-conf-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.077890 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077836    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-etc-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.077890 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.077890 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-script-lib\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-socket-dir-parent\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077915    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-multus-certs\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077931    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077947    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsrkq\" (UniqueName: \"kubernetes.io/projected/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kube-api-access-nsrkq\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077961    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-systemd-units\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.077976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysconfig\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078012    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-sys\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078039    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078053    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66b60042-598e-49bb-8de0-009bd4c1bd6b-hosts-file\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078083    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts7xc\" (UniqueName: \"kubernetes.io/projected/66b60042-598e-49bb-8de0-009bd4c1bd6b-kube-api-access-ts7xc\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-device-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-modprobe-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078164    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-multus\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078188    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-hostroot\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078231    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-multus-daemon-config\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078247    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-systemd\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078268    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7qlk\" (UniqueName: \"kubernetes.io/projected/68a4bb84-29e4-43b6-92df-7a4ea03ee820-kube-api-access-h7qlk\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b870b7-104e-44b7-b15a-8db848e34bfc-iptables-alerter-script\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078327    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078353    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-socket-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078379    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-lib-modules\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078400    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-systemd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b60042-598e-49bb-8de0-009bd4c1bd6b-tmp-dir\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078430    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-cni-binary-copy\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078481    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bngtx\" (UniqueName: \"kubernetes.io/projected/24b870b7-104e-44b7-b15a-8db848e34bfc-kube-api-access-bngtx\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078509    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-ovn\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078534    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-system-cni-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078556    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-system-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078574    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-netns\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.078939 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078589    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-kubelet\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078605    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvxcg\" (UniqueName: \"kubernetes.io/projected/b5d71931-979a-4170-abe4-d22055d75da2-kube-api-access-tvxcg\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078628    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-netns\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078653    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-env-overrides\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078676    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078692    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078715    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-host\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078743    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-dbus\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078768    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovn-node-metrics-cert\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078792    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-cnibin\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078810    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078839    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078866    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-kubernetes\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-netd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078900    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmcw7\" (UniqueName: \"kubernetes.io/projected/a2a1b468-478c-445d-af74-05dac392aec7-kube-api-access-dmcw7\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078917    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:08.079851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078940    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a3f42a4-9d45-4d56-952c-4dd04bb47261-konnectivity-ca\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078966    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-slash\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.078991    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079020    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-host\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079072    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zflwb\" (UniqueName: \"kubernetes.io/projected/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-kube-api-access-zflwb\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-cnibin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079131    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-etc-kubernetes\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079157    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-var-lib-kubelet\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079184    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-tmp\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079212    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-kubelet\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079238    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-sys-fs\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079283    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-var-lib-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079310    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-config\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079338    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-k8s-cni-cncf-io\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079364    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-tuned\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-log-socket\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079400    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-bin\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.080517 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079416    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079434    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-bin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079457    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a3f42a4-9d45-4d56-952c-4dd04bb47261-agent-certs\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079480    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xzw8\" (UniqueName: \"kubernetes.io/projected/3a1157b8-e332-4163-b20a-1edbbe85f786-kube-api-access-7xzw8\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079510    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b870b7-104e-44b7-b15a-8db848e34bfc-host-slash\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079535    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079553    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-registration-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079571    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-conf\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.081312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.079594    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-run\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.104648 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.104617    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 00:38:08.104940 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.104912    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 00:38:08.105059 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.104964    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 00:38:08.105059 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.104990    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k8zgn\""
Oct 30 00:38:08.105059 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.105007    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 00:38:08.105059 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.105037    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2bp5j\""
Oct 30 00:38:08.124481 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.124316    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kwt5m"
Oct 30 00:38:08.132469 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.132442    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kwt5m"
Oct 30 00:38:08.180218 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180111    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-etc-kubernetes\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.180218 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180162    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-var-lib-kubelet\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.180218 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180206    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-tmp\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180274    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-var-lib-kubelet\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180285    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-etc-kubernetes\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180311    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-kubelet\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180339    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-sys-fs\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180377    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-var-lib-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-config\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180427    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-k8s-cni-cncf-io\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.180471 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180453    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-tuned\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180478    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-log-socket\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-bin\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180554    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-bin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180580    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a3f42a4-9d45-4d56-952c-4dd04bb47261-agent-certs\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180608    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xzw8\" (UniqueName: \"kubernetes.io/projected/3a1157b8-e332-4163-b20a-1edbbe85f786-kube-api-access-7xzw8\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180634    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b870b7-104e-44b7-b15a-8db848e34bfc-host-slash\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180663    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180692    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-var-lib-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.180722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180690    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-registration-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180730    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-registration-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180740    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-conf\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180759    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-run\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180775    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-node-log\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180795    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zfkx\" (UniqueName: \"kubernetes.io/projected/01a7c97e-b248-4348-a07a-77652f9ce3c9-kube-api-access-8zfkx\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180775    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-bin\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180812    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-serviceca\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180832    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-kubelet-config\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180848    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-conf-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180872    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-etc-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180885    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180900    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.181022 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180921    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-k8s-cni-cncf-io\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.182760 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.182733    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-kubelet\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.182909 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.182829    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-bin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.182909 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.182838    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-log-socket\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.182909 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180664    2571 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 00:38:08.183088 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.180927    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-script-lib\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.183275 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.183251    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-run\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.183381 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.183364    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-conf\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.184823 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.184622    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-node-log\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.184823 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.184651    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-config\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.184823 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.184713    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185180    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-kubelet-config\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185247    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-socket-dir-parent\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185283    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-multus-certs\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185317    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185353    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsrkq\" (UniqueName: \"kubernetes.io/projected/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kube-api-access-nsrkq\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185387    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-systemd-units\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185416    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysconfig\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185450    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-sys\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185488    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-serviceca\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185587    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185666    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185722    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b870b7-104e-44b7-b15a-8db848e34bfc-host-slash\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.185482    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186012    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66b60042-598e-49bb-8de0-009bd4c1bd6b-hosts-file\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts7xc\" (UniqueName: \"kubernetes.io/projected/66b60042-598e-49bb-8de0-009bd4c1bd6b-kube-api-access-ts7xc\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186078    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-device-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.193303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186112    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-modprobe-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186147    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186192    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-multus\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186205    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-script-lib\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186221    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-hostroot\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186320    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-hostroot\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186373    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-socket-dir-parent\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186440    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-multus-certs\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186487    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-etc-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186527    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-sys-fs\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186548    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-systemd-units\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186611    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-sys\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186645    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-device-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186743    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-modprobe-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.186849    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186920    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-multus\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186963    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-conf-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.186965    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-tuned\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.193901 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187015    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-multus-daemon-config\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187049    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-systemd\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187095    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7qlk\" (UniqueName: \"kubernetes.io/projected/68a4bb84-29e4-43b6-92df-7a4ea03ee820-kube-api-access-h7qlk\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187111    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66b60042-598e-49bb-8de0-009bd4c1bd6b-hosts-file\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187147    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187318    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b870b7-104e-44b7-b15a-8db848e34bfc-iptables-alerter-script\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.187399    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret podName:c64e9e6d-c2b7-4520-b5d9-e8eb86720bab nodeName:}" failed. No retries permitted until 2026-10-30 00:38:08.687338511 +0000 UTC m=+3.052146146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret") pod "global-pull-secret-syncer-x2s8q" (UID: "c64e9e6d-c2b7-4520-b5d9-e8eb86720bab") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187455    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187523    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-systemd\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187761    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.187905    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b870b7-104e-44b7-b15a-8db848e34bfc-iptables-alerter-script\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188020    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysconfig\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-socket-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188090    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-multus-daemon-config\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188114    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-lib-modules\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188176    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-systemd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.194485 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188226    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-lib-modules\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188346    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-socket-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b60042-598e-49bb-8de0-009bd4c1bd6b-tmp-dir\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188487    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-cni-binary-copy\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188542    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-systemd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188639    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188681    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b60042-598e-49bb-8de0-009bd4c1bd6b-tmp-dir\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188719    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bngtx\" (UniqueName: \"kubernetes.io/projected/24b870b7-104e-44b7-b15a-8db848e34bfc-kube-api-access-bngtx\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188749    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188814    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-ovn\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188896    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-system-cni-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.188973    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-system-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189007    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-ovn\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189014    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-netns\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189047    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-kubelet\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189073    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-system-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189076    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-netns\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189109    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvxcg\" (UniqueName: \"kubernetes.io/projected/b5d71931-979a-4170-abe4-d22055d75da2-kube-api-access-tvxcg\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195070 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189118    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-cni-binary-copy\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189009    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-system-cni-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189146    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-netns\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189177    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-kubelet\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189207    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-netns\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189231    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-env-overrides\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189315    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189353    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-host\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189464    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-dbus\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189500    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189503    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovn-node-metrics-cert\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189572    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-cnibin\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189607    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189638    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-cnibin\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189684    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189719    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-kubernetes\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.195777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189723    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-host\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189767    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-netd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189811    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmcw7\" (UniqueName: \"kubernetes.io/projected/a2a1b468-478c-445d-af74-05dac392aec7-kube-api-access-dmcw7\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.189821    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189860    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.189895    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs podName:3a1157b8-e332-4163-b20a-1edbbe85f786 nodeName:}" failed. No retries permitted until 2026-10-30 00:38:08.689877233 +0000 UTC m=+3.054684874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs") pod "network-metrics-daemon-52dwv" (UID: "3a1157b8-e332-4163-b20a-1edbbe85f786") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189930    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-env-overrides\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189931    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a3f42a4-9d45-4d56-952c-4dd04bb47261-konnectivity-ca\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189980    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-slash\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.189964    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a3f42a4-9d45-4d56-952c-4dd04bb47261-agent-certs\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190003    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190049    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-slash\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190082    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-host\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190133    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zflwb\" (UniqueName: \"kubernetes.io/projected/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-kube-api-access-zflwb\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190146    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-netd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190169    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-cnibin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190306    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-cnibin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.196453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190486    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a3f42a4-9d45-4d56-952c-4dd04bb47261-konnectivity-ca\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190480    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190590    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190603    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-host\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190612    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-dbus\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.190667    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-kubernetes\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.191395    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.192941    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovn-node-metrics-cert\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.196986 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.194891    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-tmp\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.206137 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.206105    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zfkx\" (UniqueName: \"kubernetes.io/projected/01a7c97e-b248-4348-a07a-77652f9ce3c9-kube-api-access-8zfkx\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.208335 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.208304    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xzw8\" (UniqueName: \"kubernetes.io/projected/3a1157b8-e332-4163-b20a-1edbbe85f786-kube-api-access-7xzw8\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:08.208421 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.208366    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsrkq\" (UniqueName: \"kubernetes.io/projected/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kube-api-access-nsrkq\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.215736 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.215712    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:38:08.215850 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.215741    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:38:08.215850 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.215755    2571 projected.go:196] Error preparing data for projected volume kube-api-access-bc4fj for pod openshift-network-diagnostics/network-check-target-pgl9q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:08.215950 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.215875    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj podName:cd83da39-c295-436b-a07c-4c41c68de6be nodeName:}" failed. No retries permitted until 2026-10-30 00:38:08.715832952 +0000 UTC m=+3.080640573 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bc4fj" (UniqueName: "kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj") pod "network-check-target-pgl9q" (UID: "cd83da39-c295-436b-a07c-4c41c68de6be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:08.218216 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.218184    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7qlk\" (UniqueName: \"kubernetes.io/projected/68a4bb84-29e4-43b6-92df-7a4ea03ee820-kube-api-access-h7qlk\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.219477 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.219450    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts7xc\" (UniqueName: \"kubernetes.io/projected/66b60042-598e-49bb-8de0-009bd4c1bd6b-kube-api-access-ts7xc\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.219919 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.219897    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zflwb\" (UniqueName: \"kubernetes.io/projected/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-kube-api-access-zflwb\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.221477 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.221452    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bngtx\" (UniqueName: \"kubernetes.io/projected/24b870b7-104e-44b7-b15a-8db848e34bfc-kube-api-access-bngtx\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.221570 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.221461    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmcw7\" (UniqueName: \"kubernetes.io/projected/a2a1b468-478c-445d-af74-05dac392aec7-kube-api-access-dmcw7\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.223652 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.223627    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvxcg\" (UniqueName: \"kubernetes.io/projected/b5d71931-979a-4170-abe4-d22055d75da2-kube-api-access-tvxcg\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.294350 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.294314    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:38:08.305059 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.305034    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:38:08.305953 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.305919    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66b60042_598e_49bb_8de0_009bd4c1bd6b.slice/crio-ba4a7524aa24ad2414db1fc9a78d51a2c0ef6ea199b826bebfa9a28c0d92e11d WatchSource:0}: Error finding container ba4a7524aa24ad2414db1fc9a78d51a2c0ef6ea199b826bebfa9a28c0d92e11d: Status 404 returned error can't find the container with id ba4a7524aa24ad2414db1fc9a78d51a2c0ef6ea199b826bebfa9a28c0d92e11d
Oct 30 00:38:08.314242 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.314196    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ddf5d"
Oct 30 00:38:08.314686 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.314657    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68a4bb84_29e4_43b6_92df_7a4ea03ee820.slice/crio-c12ddaa8c7d3932f3b903e288850f5ac99631b851a639589335bd335d8cb1cdc WatchSource:0}: Error finding container c12ddaa8c7d3932f3b903e288850f5ac99631b851a639589335bd335d8cb1cdc: Status 404 returned error can't find the container with id c12ddaa8c7d3932f3b903e288850f5ac99631b851a639589335bd335d8cb1cdc
Oct 30 00:38:08.322915 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.322704    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:38:08.325622 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.325584    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d71931_979a_4170_abe4_d22055d75da2.slice/crio-d501239df6e20428a1af1daef082cb9622bcff5d5d981d348a66970a6e53509f WatchSource:0}: Error finding container d501239df6e20428a1af1daef082cb9622bcff5d5d981d348a66970a6e53509f: Status 404 returned error can't find the container with id d501239df6e20428a1af1daef082cb9622bcff5d5d981d348a66970a6e53509f
Oct 30 00:38:08.341357 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.341329    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:08.344415 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.344390    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:38:08.349341 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.349317    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:08.354564 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.354511    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01a7c97e_b248_4348_a07a_77652f9ce3c9.slice/crio-412046e464fe9bfdfcf1446b319fce0f33520fece5e515c677d3dac5e53c044d WatchSource:0}: Error finding container 412046e464fe9bfdfcf1446b319fce0f33520fece5e515c677d3dac5e53c044d: Status 404 returned error can't find the container with id 412046e464fe9bfdfcf1446b319fce0f33520fece5e515c677d3dac5e53c044d
Oct 30 00:38:08.358440 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.358411    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:38:08.359215 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.359126    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a3f42a4_9d45_4d56_952c_4dd04bb47261.slice/crio-fa0428cff534eb96c9e1ccb45c7c8f2fcceccb04f1d581b08392a6aedd113040 WatchSource:0}: Error finding container fa0428cff534eb96c9e1ccb45c7c8f2fcceccb04f1d581b08392a6aedd113040: Status 404 returned error can't find the container with id fa0428cff534eb96c9e1ccb45c7c8f2fcceccb04f1d581b08392a6aedd113040
Oct 30 00:38:08.359967 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.359943    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f8e2990_d9f0_41a8_a8d6_09c67b54e9f3.slice/crio-26383512cc36d898f8de6229f9f4f6830fcac06d7fba3ffa229293be41790096 WatchSource:0}: Error finding container 26383512cc36d898f8de6229f9f4f6830fcac06d7fba3ffa229293be41790096: Status 404 returned error can't find the container with id 26383512cc36d898f8de6229f9f4f6830fcac06d7fba3ffa229293be41790096
Oct 30 00:38:08.369600 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.369574    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57d0d345_d190_4a7b_b61c_f2f2da64ffc8.slice/crio-956ecf285f0d4ec7a0a2695c9b7e754359909d3deee4b6bf9f3d12408428c4b8 WatchSource:0}: Error finding container 956ecf285f0d4ec7a0a2695c9b7e754359909d3deee4b6bf9f3d12408428c4b8: Status 404 returned error can't find the container with id 956ecf285f0d4ec7a0a2695c9b7e754359909d3deee4b6bf9f3d12408428c4b8
Oct 30 00:38:08.371100 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.371079    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:38:08.382329 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:08.382302    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2a1b468_478c_445d_af74_05dac392aec7.slice/crio-9388184f99755682042b0e2b0aa92546c6e7f41c3ebc22d89e6ef0c6f3789ff3 WatchSource:0}: Error finding container 9388184f99755682042b0e2b0aa92546c6e7f41c3ebc22d89e6ef0c6f3789ff3: Status 404 returned error can't find the container with id 9388184f99755682042b0e2b0aa92546c6e7f41c3ebc22d89e6ef0c6f3789ff3
Oct 30 00:38:08.694170 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.694077    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:08.694170 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.694145    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:08.694434 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.694347    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:08.694434 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.694414    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs podName:3a1157b8-e332-4163-b20a-1edbbe85f786 nodeName:}" failed. No retries permitted until 2026-10-30 00:38:09.69439283 +0000 UTC m=+4.059200449 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs") pod "network-metrics-daemon-52dwv" (UID: "3a1157b8-e332-4163-b20a-1edbbe85f786") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:08.694800 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.694784    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:08.694857 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.694841    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret podName:c64e9e6d-c2b7-4520-b5d9-e8eb86720bab nodeName:}" failed. No retries permitted until 2026-10-30 00:38:09.694824113 +0000 UTC m=+4.059631718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret") pod "global-pull-secret-syncer-x2s8q" (UID: "c64e9e6d-c2b7-4520-b5d9-e8eb86720bab") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:08.794970 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:08.794927    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:08.795161 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.795140    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:38:08.795239 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.795176    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:38:08.795239 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.795189    2571 projected.go:196] Error preparing data for projected volume kube-api-access-bc4fj for pod openshift-network-diagnostics/network-check-target-pgl9q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:08.795378 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:08.795277    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj podName:cd83da39-c295-436b-a07c-4c41c68de6be nodeName:}" failed. No retries permitted until 2026-10-30 00:38:09.795256926 +0000 UTC m=+4.160064543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bc4fj" (UniqueName: "kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj") pod "network-check-target-pgl9q" (UID: "cd83da39-c295-436b-a07c-4c41c68de6be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:09.093129 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.093086    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:09.094084 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.093261    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:09.127086 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.126538    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal" event={"ID":"7a3c183736bfb0e3d8475eab25325f35","Type":"ContainerStarted","Data":"eba591903716040e75bb9b7d66c5513098ce878f2e97c368f8f8792e4322da5f"}
Oct 30 00:38:09.133092 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.133052    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6srk" event={"ID":"57d0d345-d190-4a7b-b61c-f2f2da64ffc8","Type":"ContainerStarted","Data":"956ecf285f0d4ec7a0a2695c9b7e754359909d3deee4b6bf9f3d12408428c4b8"}
Oct 30 00:38:09.134503 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.134436    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 00:33:08 +0000 UTC" deadline="2028-08-05 02:25:19.96087714 +0000 UTC"
Oct 30 00:38:09.134503 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.134501    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15481h47m10.826379389s"
Oct 30 00:38:09.136590 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.136560    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2" event={"ID":"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3","Type":"ContainerStarted","Data":"26383512cc36d898f8de6229f9f4f6830fcac06d7fba3ffa229293be41790096"}
Oct 30 00:38:09.139962 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.139934    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"412046e464fe9bfdfcf1446b319fce0f33520fece5e515c677d3dac5e53c044d"}
Oct 30 00:38:09.142693 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.142667    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m6vlx" event={"ID":"24b870b7-104e-44b7-b15a-8db848e34bfc","Type":"ContainerStarted","Data":"3cc4adb2181f5cb23a4bba9093aaa1d5108e18d8038a46f406bb53f9ecbbcb29"}
Oct 30 00:38:09.157884 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.157806    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ddf5d" event={"ID":"b5d71931-979a-4170-abe4-d22055d75da2","Type":"ContainerStarted","Data":"d501239df6e20428a1af1daef082cb9622bcff5d5d981d348a66970a6e53509f"}
Oct 30 00:38:09.161688 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.161612    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5sv2w" event={"ID":"68a4bb84-29e4-43b6-92df-7a4ea03ee820","Type":"ContainerStarted","Data":"c12ddaa8c7d3932f3b903e288850f5ac99631b851a639589335bd335d8cb1cdc"}
Oct 30 00:38:09.164659 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.164569    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ql49r" event={"ID":"66b60042-598e-49bb-8de0-009bd4c1bd6b","Type":"ContainerStarted","Data":"ba4a7524aa24ad2414db1fc9a78d51a2c0ef6ea199b826bebfa9a28c0d92e11d"}
Oct 30 00:38:09.167784 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.167732    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal" podStartSLOduration=2.167716047 podStartE2EDuration="2.167716047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:09.167157873 +0000 UTC m=+3.531965503" watchObservedRunningTime="2026-10-30 00:38:09.167716047 +0000 UTC m=+3.532523677"
Oct 30 00:38:09.184351 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.184324    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerStarted","Data":"9388184f99755682042b0e2b0aa92546c6e7f41c3ebc22d89e6ef0c6f3789ff3"}
Oct 30 00:38:09.193053 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.193022    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p8fxn" event={"ID":"5a3f42a4-9d45-4d56-952c-4dd04bb47261","Type":"ContainerStarted","Data":"fa0428cff534eb96c9e1ccb45c7c8f2fcceccb04f1d581b08392a6aedd113040"}
Oct 30 00:38:09.706491 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.706455    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:09.706633 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.706515    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:09.706676 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.706650    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:09.706734 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.706709    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs podName:3a1157b8-e332-4163-b20a-1edbbe85f786 nodeName:}" failed. No retries permitted until 2026-10-30 00:38:11.706691064 +0000 UTC m=+6.071498684 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs") pod "network-metrics-daemon-52dwv" (UID: "3a1157b8-e332-4163-b20a-1edbbe85f786") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:09.707055 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.707036    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:09.707118 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.707071    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret podName:c64e9e6d-c2b7-4520-b5d9-e8eb86720bab nodeName:}" failed. No retries permitted until 2026-10-30 00:38:11.707062524 +0000 UTC m=+6.071870140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret") pod "global-pull-secret-syncer-x2s8q" (UID: "c64e9e6d-c2b7-4520-b5d9-e8eb86720bab") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:09.807932 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:09.807428    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:09.807932 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.807560    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:38:09.807932 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.807576    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:38:09.807932 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.807585    2571 projected.go:196] Error preparing data for projected volume kube-api-access-bc4fj for pod openshift-network-diagnostics/network-check-target-pgl9q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:09.807932 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:09.807627    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj podName:cd83da39-c295-436b-a07c-4c41c68de6be nodeName:}" failed. No retries permitted until 2026-10-30 00:38:11.807615654 +0000 UTC m=+6.172423256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bc4fj" (UniqueName: "kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj") pod "network-check-target-pgl9q" (UID: "cd83da39-c295-436b-a07c-4c41c68de6be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:10.094589 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:10.094338    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:10.095037 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:10.094420    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:10.095037 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:10.094710    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:10.095037 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:10.094801    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:10.136097 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:10.136047    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 00:33:08 +0000 UTC" deadline="2028-04-12 08:34:44.962037143 +0000 UTC"
Oct 30 00:38:10.136097 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:10.136086    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12727h56m34.825956226s"
Oct 30 00:38:11.093675 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:11.093637    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:11.093918 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.093756    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:11.205521 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:11.205478    2571 generic.go:356] "Generic (PLEG): container finished" podID="3e516b0f6958377721da4db26ec47f8e" containerID="008008a36334aec87de5602740b5dd59f614014d70225c56318b41f40a62da89" exitCode=0
Oct 30 00:38:11.205877 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:11.205542    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal" event={"ID":"3e516b0f6958377721da4db26ec47f8e","Type":"ContainerDied","Data":"008008a36334aec87de5602740b5dd59f614014d70225c56318b41f40a62da89"}
Oct 30 00:38:11.725941 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:11.725104    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:11.725941 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:11.725164    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:11.725941 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.725348    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:11.725941 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.725415    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs podName:3a1157b8-e332-4163-b20a-1edbbe85f786 nodeName:}" failed. No retries permitted until 2026-10-30 00:38:15.725395921 +0000 UTC m=+10.090203547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs") pod "network-metrics-daemon-52dwv" (UID: "3a1157b8-e332-4163-b20a-1edbbe85f786") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:11.725941 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.725803    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:11.725941 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.725854    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret podName:c64e9e6d-c2b7-4520-b5d9-e8eb86720bab nodeName:}" failed. No retries permitted until 2026-10-30 00:38:15.725839358 +0000 UTC m=+10.090646975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret") pod "global-pull-secret-syncer-x2s8q" (UID: "c64e9e6d-c2b7-4520-b5d9-e8eb86720bab") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:11.827290 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:11.826543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:11.827290 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.826715    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:38:11.827290 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.826737    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:38:11.827290 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.826751    2571 projected.go:196] Error preparing data for projected volume kube-api-access-bc4fj for pod openshift-network-diagnostics/network-check-target-pgl9q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:11.827290 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:11.826811    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj podName:cd83da39-c295-436b-a07c-4c41c68de6be nodeName:}" failed. No retries permitted until 2026-10-30 00:38:15.826791728 +0000 UTC m=+10.191599348 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bc4fj" (UniqueName: "kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj") pod "network-check-target-pgl9q" (UID: "cd83da39-c295-436b-a07c-4c41c68de6be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:12.098429 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:12.095403    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:12.098429 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:12.095574    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:12.098429 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:12.095677    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:12.098429 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:12.095839    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:12.212132 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:12.212094    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal" event={"ID":"3e516b0f6958377721da4db26ec47f8e","Type":"ContainerStarted","Data":"b9ea781b817e79e507d5a5c0170e0dfc6026d0ef69eee03ac611f219e838002b"}
Oct 30 00:38:12.216348 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:12.216281    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="c062b6a4821c0d56074e2f3bbacfb7c922724255567a84b9fdaf02c2941f040c" exitCode=0
Oct 30 00:38:12.216475 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:12.216375    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerDied","Data":"c062b6a4821c0d56074e2f3bbacfb7c922724255567a84b9fdaf02c2941f040c"}
Oct 30 00:38:12.303681 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:12.303621    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal" podStartSLOduration=5.303599988 podStartE2EDuration="5.303599988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:12.25330459 +0000 UTC m=+6.618112216" watchObservedRunningTime="2026-10-30 00:38:12.303599988 +0000 UTC m=+6.668407616"
Oct 30 00:38:13.094320 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:13.093782    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:13.094320 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:13.093919    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:13.225191 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:13.225103    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p8fxn" event={"ID":"5a3f42a4-9d45-4d56-952c-4dd04bb47261","Type":"ContainerStarted","Data":"c0cbccc22ce5a407f91ab6e4bae3de56de931be781d22fb67ea4cc5a6d137055"}
Oct 30 00:38:14.096742 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:14.096036    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:14.096742 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:14.096189    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:14.096742 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:14.096582    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:14.096742 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:14.096689    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:14.228919 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:14.228879    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6srk" event={"ID":"57d0d345-d190-4a7b-b61c-f2f2da64ffc8","Type":"ContainerStarted","Data":"6d5c506cb61bacbc300035bc63ccd83983d46df73bb43c5b91bb639184d7a32d"}
Oct 30 00:38:14.269959 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:14.269896    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p8fxn" podStartSLOduration=4.444226522 podStartE2EDuration="8.269874889s" totalImagesPullingTime="3.825648367s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:13.264384617 +0000 UTC m=+7.629192242" watchObservedRunningTime="2026-10-30 00:38:14.269874889 +0000 UTC m=+8.634682516"
Oct 30 00:38:15.093528 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:15.092969    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:15.093528 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.093106    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:15.234664 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:15.234545    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2" event={"ID":"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3","Type":"ContainerStarted","Data":"cf8757224f9b7f410a68a95db528b5a86b1f4e4061898c67e1af0ab888970b02"}
Oct 30 00:38:15.765979 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:15.765938    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:15.765979 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:15.765985    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:15.766196 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.766087    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:15.766196 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.766151    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs podName:3a1157b8-e332-4163-b20a-1edbbe85f786 nodeName:}" failed. No retries permitted until 2026-10-30 00:38:23.766132969 +0000 UTC m=+18.130940575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs") pod "network-metrics-daemon-52dwv" (UID: "3a1157b8-e332-4163-b20a-1edbbe85f786") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:15.766675 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.766580    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:15.766675 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.766632    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret podName:c64e9e6d-c2b7-4520-b5d9-e8eb86720bab nodeName:}" failed. No retries permitted until 2026-10-30 00:38:23.76661804 +0000 UTC m=+18.131425656 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret") pod "global-pull-secret-syncer-x2s8q" (UID: "c64e9e6d-c2b7-4520-b5d9-e8eb86720bab") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:15.867081 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:15.866765    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:15.867081 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.866957    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:38:15.867081 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.866977    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:38:15.867081 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.866990    2571 projected.go:196] Error preparing data for projected volume kube-api-access-bc4fj for pod openshift-network-diagnostics/network-check-target-pgl9q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:15.867081 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:15.867041    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj podName:cd83da39-c295-436b-a07c-4c41c68de6be nodeName:}" failed. No retries permitted until 2026-10-30 00:38:23.867022802 +0000 UTC m=+18.231830429 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bc4fj" (UniqueName: "kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj") pod "network-check-target-pgl9q" (UID: "cd83da39-c295-436b-a07c-4c41c68de6be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:16.026220 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.026047    2571 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 00:38:16.043459 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.041484    2571 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T00:38:16.026076822Z","UUID":"494b8e2c-158c-4e73-be8d-e25b93b190b4","Handler":null,"Name":"","Endpoint":""}
Oct 30 00:38:16.046693 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.046384    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 00:38:16.046693 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.046429    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 00:38:16.096508 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.096469    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:16.096678 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:16.096591    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:16.097013 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.096992    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:16.097226 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:16.097203    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:16.239688 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.239531    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ddf5d" event={"ID":"b5d71931-979a-4170-abe4-d22055d75da2","Type":"ContainerStarted","Data":"a864cc80ce7b2d5db6ab843c5bb4876bcce73db8988e02c609581c98ba6ea595"}
Oct 30 00:38:16.242513 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.242480    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2" event={"ID":"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3","Type":"ContainerStarted","Data":"321b4047858947d9d8522ea4ae117ad815eb478925181bff88ac13fd9ca741b3"}
Oct 30 00:38:16.280394 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.280299    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g6srk" podStartSLOduration=5.322826405 podStartE2EDuration="10.280282826s" totalImagesPullingTime="4.957456421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:14.270720319 +0000 UTC m=+8.635527945" watchObservedRunningTime="2026-10-30 00:38:16.280282826 +0000 UTC m=+10.645090456"
Oct 30 00:38:16.457910 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.457879    2571 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:16.458789 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.458767    2571 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:16.495144 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:16.495100    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-ddf5d" podStartSLOduration=2.967369659 podStartE2EDuration="10.495086531s" totalImagesPullingTime="7.527716872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:16.281416001 +0000 UTC m=+10.646223627" watchObservedRunningTime="2026-10-30 00:38:16.495086531 +0000 UTC m=+10.859894150"
Oct 30 00:38:17.093546 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:17.093323    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:17.093985 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:17.093657    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:17.248058 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:17.248020    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ql49r" event={"ID":"66b60042-598e-49bb-8de0-009bd4c1bd6b","Type":"ContainerStarted","Data":"121bdcf17e0523edc73863bf9f3c277396d9d82657cd83f3e0dc6663ecedd39e"}
Oct 30 00:38:17.250402 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:17.250366    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="d1e108341e402f7270013ca2f166de6c7b8909b2955d067fbcb4874193145e29" exitCode=0
Oct 30 00:38:17.250539 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:17.250467    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerDied","Data":"d1e108341e402f7270013ca2f166de6c7b8909b2955d067fbcb4874193145e29"}
Oct 30 00:38:17.255264 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:17.255239    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2" event={"ID":"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3","Type":"ContainerStarted","Data":"e7983efdc5682545a41d6f9ef7c87592379c23378e9eb337baa61304a888f797"}
Oct 30 00:38:17.285612 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:17.285564    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-ql49r" podStartSLOduration=3.28721753 podStartE2EDuration="11.285551287s" totalImagesPullingTime="7.998333757s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:17.285079197 +0000 UTC m=+11.649886824" watchObservedRunningTime="2026-10-30 00:38:17.285551287 +0000 UTC m=+11.650358911"
Oct 30 00:38:17.324310 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:17.324245    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2" podStartSLOduration=3.073820726 podStartE2EDuration="11.324226618s" totalImagesPullingTime="8.250405892s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:17.323684986 +0000 UTC m=+11.688492610" watchObservedRunningTime="2026-10-30 00:38:17.324226618 +0000 UTC m=+11.689034246"
Oct 30 00:38:18.093936 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:18.093907    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:18.094089 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:18.094033    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:18.094233 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:18.094216    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:18.094339 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:18.094313    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:18.258457 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:18.258377    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-m6vlx" event={"ID":"24b870b7-104e-44b7-b15a-8db848e34bfc","Type":"ContainerStarted","Data":"d2a7c1addb0ae5180bc7936a86d43c3d91f7b8a29262f40ce4b7bcc632513bed"}
Oct 30 00:38:18.261261 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:18.261228    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="c5c5841ba070dedc1ec6fa9850f6e4df9ff42707a2fba98a8c3e029787f829f9" exitCode=0
Oct 30 00:38:18.261376 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:18.261261    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerDied","Data":"c5c5841ba070dedc1ec6fa9850f6e4df9ff42707a2fba98a8c3e029787f829f9"}
Oct 30 00:38:18.297605 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:18.297546    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-m6vlx" podStartSLOduration=4.183839505 podStartE2EDuration="12.297529367s" totalImagesPullingTime="8.113689862s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:18.297011905 +0000 UTC m=+12.661819529" watchObservedRunningTime="2026-10-30 00:38:18.297529367 +0000 UTC m=+12.662336992"
Oct 30 00:38:19.093006 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:19.092984    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:19.093127 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:19.093093    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:19.266599 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:19.266513    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="4098c8cd7475f842a973c8b4b153bc570fdbb1b408155fbe1c282738c76d80cb" exitCode=0
Oct 30 00:38:19.267293 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:19.266604    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerDied","Data":"4098c8cd7475f842a973c8b4b153bc570fdbb1b408155fbe1c282738c76d80cb"}
Oct 30 00:38:19.268560 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:19.268524    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5sv2w" event={"ID":"68a4bb84-29e4-43b6-92df-7a4ea03ee820","Type":"ContainerStarted","Data":"e7ad63ed96796cd8f3ed3e8b31082084526381bf4fabb97bdbe0886eef09e7a7"}
Oct 30 00:38:19.349053 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:19.349000    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5sv2w" podStartSLOduration=3.008400354 podStartE2EDuration="13.348986811s" totalImagesPullingTime="10.340586457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:19.348478909 +0000 UTC m=+13.713286545" watchObservedRunningTime="2026-10-30 00:38:19.348986811 +0000 UTC m=+13.713794504"
Oct 30 00:38:20.094542 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:20.094513    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:20.094542 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:20.094529    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:20.094772 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:20.094637    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:20.094772 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:20.094715    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:21.093033 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:21.092996    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:21.093664 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:21.093129    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:22.093777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:22.093741    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:22.094288 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:22.093741    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:22.094288 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:22.093892    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:22.094288 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:22.093969    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:22.278478 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:22.278441    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="7704e2f2d4cd45f0c716f6a0d1c7616a71e9e696b85654968401f543a1bb0b6b" exitCode=0
Oct 30 00:38:22.278666 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:22.278495    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerDied","Data":"7704e2f2d4cd45f0c716f6a0d1c7616a71e9e696b85654968401f543a1bb0b6b"}
Oct 30 00:38:23.093356 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.093154    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:23.093468 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.093441    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:23.283033 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.282999    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"f3fbafced4804ed0272592e49ee681a56d1fea1ccc2d377a8604cd590d48282a"}
Oct 30 00:38:23.283498 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.283099    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"1056388921a2c2668bd9777c76e8b2ad0b7b46e7dedc22fc372958814d240710"}
Oct 30 00:38:23.283498 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.283143    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"9f8f2d2aad2be413ed41dcaa9becc90dbf7655460597d23633fde4f1b45bc885"}
Oct 30 00:38:23.283498 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.283203    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"25dc1767a16573c93ffea5abaf3273661c3aea79f1a2f475409589bc3bc5fb7b"}
Oct 30 00:38:23.283498 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.283248    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"f1e6e880020e3ea6ff69e5235848fb8be018f5de9df5799f09c024f46214d412"}
Oct 30 00:38:23.283498 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.283297    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"87c283d2e90c09a4eae15652c513a13c93d4e27adf299426c4b3d7d9545341d6"}
Oct 30 00:38:23.286093 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.286067    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="0648bbe07e2f1fe8df135eaead88e8c8e7291cbde961cc9e72fdf7bdb4e14767" exitCode=0
Oct 30 00:38:23.286246 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.286115    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerDied","Data":"0648bbe07e2f1fe8df135eaead88e8c8e7291cbde961cc9e72fdf7bdb4e14767"}
Oct 30 00:38:23.823509 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.823320    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:23.823675 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.823521    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:23.823675 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.823464    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:23.823675 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.823624    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:23.823675 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.823649    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret podName:c64e9e6d-c2b7-4520-b5d9-e8eb86720bab nodeName:}" failed. No retries permitted until 2026-10-30 00:38:39.823626006 +0000 UTC m=+34.188433609 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret") pod "global-pull-secret-syncer-x2s8q" (UID: "c64e9e6d-c2b7-4520-b5d9-e8eb86720bab") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:23.823675 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.823667    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs podName:3a1157b8-e332-4163-b20a-1edbbe85f786 nodeName:}" failed. No retries permitted until 2026-10-30 00:38:39.823657291 +0000 UTC m=+34.188464894 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs") pod "network-metrics-daemon-52dwv" (UID: "3a1157b8-e332-4163-b20a-1edbbe85f786") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:23.924803 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:23.924751    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:23.924982 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.924917    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:38:23.924982 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.924943    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:38:23.924982 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.924953    2571 projected.go:196] Error preparing data for projected volume kube-api-access-bc4fj for pod openshift-network-diagnostics/network-check-target-pgl9q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:23.925068 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:23.925002    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj podName:cd83da39-c295-436b-a07c-4c41c68de6be nodeName:}" failed. No retries permitted until 2026-10-30 00:38:39.924987875 +0000 UTC m=+34.289795479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bc4fj" (UniqueName: "kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj") pod "network-check-target-pgl9q" (UID: "cd83da39-c295-436b-a07c-4c41c68de6be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:24.093468 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:24.093374    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:24.093468 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:24.093405    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:24.093686 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:24.093481    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:24.093686 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:24.093633    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:24.292003 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:24.291964    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" event={"ID":"a2a1b468-478c-445d-af74-05dac392aec7","Type":"ContainerStarted","Data":"5c20f08f8a95cf1a470b66d4f7308115d6806269b7c4685f20dd10bb3bf4b814"}
Oct 30 00:38:24.335710 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:24.335657    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ckp4t" podStartSLOduration=7.975226982 podStartE2EDuration="18.335644488s" totalImagesPullingTime="10.360417506s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:24.335412299 +0000 UTC m=+18.700219923" watchObservedRunningTime="2026-10-30 00:38:24.335644488 +0000 UTC m=+18.700452113"
Oct 30 00:38:25.093870 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:25.093834    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:25.094071 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:25.093962    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:25.298416 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:25.298374    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"930f41dbe71c0e8d065effbba3772da17ff8bbc2005f0de0ee15640baf7dffec"}
Oct 30 00:38:26.094156 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:26.094131    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:26.094337 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:26.094260    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:26.094337 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:26.094281    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:26.094420 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:26.094379    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:27.093514 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:27.093479    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:27.094003 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:27.093588    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:28.093677 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:28.093634    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:28.094260 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:28.093689    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:28.094260 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:28.093815    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:28.094260 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:28.093942    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:28.309863 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:28.309823    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" event={"ID":"01a7c97e-b248-4348-a07a-77652f9ce3c9","Type":"ContainerStarted","Data":"71c747c2b841d13437a0e1219ad3984c387379ed608285b9857c9c913c00e3a0"}
Oct 30 00:38:28.310131 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:28.310105    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:28.325969 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:28.325937    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:28.396987 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:28.396881    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt" podStartSLOduration=7.898353099 podStartE2EDuration="22.396869187s" totalImagesPullingTime="14.498516088s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:28.396542582 +0000 UTC m=+22.761350208" watchObservedRunningTime="2026-10-30 00:38:28.396869187 +0000 UTC m=+22.761676813"
Oct 30 00:38:29.093641 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.093613    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:29.093779 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:29.093760    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:29.312516 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.312274    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:29.318810 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.318786    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:29.333605 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.333314    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:38:29.362080 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.361975    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-52dwv"]
Oct 30 00:38:29.362247 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.362114    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:29.362308 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:29.362284    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:29.363205 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.363161    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x2s8q"]
Oct 30 00:38:29.363321 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.363284    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:29.363377 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:29.363361    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:29.363734 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.363717    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pgl9q"]
Oct 30 00:38:29.363810 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:29.363799    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:29.363888 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:29.363872    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:31.093767 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:31.093727    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:31.093767 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:31.093755    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:31.094238 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:31.093755    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:31.094238 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:31.093853    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:31.094238 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:31.093914    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:31.094238 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:31.093977    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:33.093479 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:33.093445    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:33.094029 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:33.093489    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:33.094029 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:33.093440    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:33.094029 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:33.093570    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:33.094029 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:33.093646    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:33.094029 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:33.093699    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:35.093247 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:35.093206    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:35.093247 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:35.093245    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:35.093738 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:35.093247    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:35.093738 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:35.093320    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:35.093738 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:35.093433    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:35.093738 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:35.093540    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:36.690309 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:36.690269    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:36.690722 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:36.690393    2571 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:38:36.691112 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:36.691087    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:38:37.006307 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:37.006232    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ql49r_66b60042-598e-49bb-8de0-009bd4c1bd6b/dns-node-resolver/0.log"
Oct 30 00:38:37.093125 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:37.093086    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:37.093340 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:37.093168    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:37.093340 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:37.093197    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:37.093340 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:37.093295    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:37.093504 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:37.093410    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:37.093504 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:37.093485    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:38.146326 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:38.146301    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6srk_57d0d345-d190-4a7b-b61c-f2f2da64ffc8/node-ca/0.log"
Oct 30 00:38:39.093218 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:39.093193    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:39.093361 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:39.093225    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:39.093361 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:39.093294    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:39.093361 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.093312    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:39.093517 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.093365    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:39.093517 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.093431    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:39.837014 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:39.836981    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:39.837014 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:39.837020    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:39.837537 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.837461    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:39.837537 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.837535    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs podName:3a1157b8-e332-4163-b20a-1edbbe85f786 nodeName:}" failed. No retries permitted until 2026-10-30 00:39:11.837515761 +0000 UTC m=+66.202323381 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs") pod "network-metrics-daemon-52dwv" (UID: "3a1157b8-e332-4163-b20a-1edbbe85f786") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 00:38:39.837981 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.837947    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:39.839958 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.839934    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret podName:c64e9e6d-c2b7-4520-b5d9-e8eb86720bab nodeName:}" failed. No retries permitted until 2026-10-30 00:39:11.83991841 +0000 UTC m=+66.204726013 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret") pod "global-pull-secret-syncer-x2s8q" (UID: "c64e9e6d-c2b7-4520-b5d9-e8eb86720bab") : object "kube-system"/"original-pull-secret" not registered
Oct 30 00:38:39.938318 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:39.938289    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:39.938478 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.938423    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 00:38:39.938478 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.938436    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 00:38:39.938478 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.938445    2571 projected.go:196] Error preparing data for projected volume kube-api-access-bc4fj for pod openshift-network-diagnostics/network-check-target-pgl9q: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:39.938574 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:39.938487    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj podName:cd83da39-c295-436b-a07c-4c41c68de6be nodeName:}" failed. No retries permitted until 2026-10-30 00:39:11.938475907 +0000 UTC m=+66.303283524 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bc4fj" (UniqueName: "kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj") pod "network-check-target-pgl9q" (UID: "cd83da39-c295-436b-a07c-4c41c68de6be") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 00:38:41.093255 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:41.093222    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:41.093255 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:41.093248    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:41.093720 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:41.093221    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:41.093720 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:41.093335    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:41.093720 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:41.093409    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:41.093720 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:41.093472    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:43.093122 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:43.093086    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:43.093122 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:43.093118    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:43.093570 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:43.093216    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:43.093570 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:43.093264    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:43.093570 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:43.093343    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:43.093570 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:43.093382    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:45.093113 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:45.093073    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:45.093113 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:45.093103    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:45.093113 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:45.093115    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:45.093708 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:45.093221    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:45.093708 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:45.093290    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:45.093708 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:45.093377    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:47.093037 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:47.093002    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:47.093442 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:47.093081    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:47.093442 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:47.093167    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:47.093442 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:47.093206    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:47.093442 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:47.093265    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:47.093442 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:47.093327    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:49.092894 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:49.092856    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:49.092894 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:49.092893    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:49.093448 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:49.092890    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:49.093448 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:49.092978    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:49.093448 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:49.093071    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:49.093448 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:49.093123    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:51.093377 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.093342    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:51.093884 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.093383    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:51.093884 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.093449    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:51.093884 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:51.093468    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x2s8q" podUID="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab"
Oct 30 00:38:51.093884 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:51.093535    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-52dwv" podUID="3a1157b8-e332-4163-b20a-1edbbe85f786"
Oct 30 00:38:51.093884 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:38:51.093589    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pgl9q" podUID="cd83da39-c295-436b-a07c-4c41c68de6be"
Oct 30 00:38:51.389155 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.389077    2571 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeReady"
Oct 30 00:38:51.389326 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.389227    2571 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 00:38:51.463959 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.463925    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qlw4m"]
Oct 30 00:38:51.465656 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.465637    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.467384 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.467363    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-drhmz"]
Oct 30 00:38:51.468949 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.468932    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.484083 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.484063    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 00:38:51.484221 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.484203    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 00:38:51.484277 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.484240    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7gr5j\""
Oct 30 00:38:51.492564 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.492543    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-7zp52"]
Oct 30 00:38:51.494296 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.494276    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qlw4m"]
Oct 30 00:38:51.494395 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.494319    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-drhmz"]
Oct 30 00:38:51.494453 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.494401    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:51.515755 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.515728    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7zp52"]
Oct 30 00:38:51.583049 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583019    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 00:38:51.583262 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583114    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 00:38:51.583262 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583161    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 00:38:51.583262 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583210    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 00:38:51.583262 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583251    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 00:38:51.583437 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583425    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 00:38:51.583478 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583426    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jxbz5\""
Oct 30 00:38:51.583478 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583443    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 00:38:51.583537 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.583491    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bqzrd\""
Oct 30 00:38:51.618275 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618248    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb95c197-005f-4deb-b563-107e7e0aff33-config-volume\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.618400 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb95c197-005f-4deb-b563-107e7e0aff33-tmp-dir\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.618400 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618299    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d14ce64-f501-4c56-aeff-137f931bfea9-data-volume\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.618400 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618317    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpqk8\" (UniqueName: \"kubernetes.io/projected/fb95c197-005f-4deb-b563-107e7e0aff33-kube-api-access-jpqk8\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.618400 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618361    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndnng\" (UniqueName: \"kubernetes.io/projected/6ebd1a13-21bc-4de8-aaad-4e289981ee10-kube-api-access-ndnng\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:51.618521 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618421    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb95c197-005f-4deb-b563-107e7e0aff33-metrics-tls\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.618521 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618464    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d14ce64-f501-4c56-aeff-137f931bfea9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.618580 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618534    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d14ce64-f501-4c56-aeff-137f931bfea9-crio-socket\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.618580 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618551    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.618639 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618590    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7dcp\" (UniqueName: \"kubernetes.io/projected/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-api-access-h7dcp\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.618673 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.618634    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ebd1a13-21bc-4de8-aaad-4e289981ee10-cert\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:51.719663 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.719577    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7dcp\" (UniqueName: \"kubernetes.io/projected/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-api-access-h7dcp\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.719663 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.719628    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ebd1a13-21bc-4de8-aaad-4e289981ee10-cert\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:51.719899 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.719879    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb95c197-005f-4deb-b563-107e7e0aff33-config-volume\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.719934 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.719915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb95c197-005f-4deb-b563-107e7e0aff33-tmp-dir\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.719979 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.719938    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d14ce64-f501-4c56-aeff-137f931bfea9-data-volume\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.719979 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.719954    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpqk8\" (UniqueName: \"kubernetes.io/projected/fb95c197-005f-4deb-b563-107e7e0aff33-kube-api-access-jpqk8\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.719979 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.719969    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndnng\" (UniqueName: \"kubernetes.io/projected/6ebd1a13-21bc-4de8-aaad-4e289981ee10-kube-api-access-ndnng\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:51.720121 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720004    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb95c197-005f-4deb-b563-107e7e0aff33-metrics-tls\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.720121 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720056    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d14ce64-f501-4c56-aeff-137f931bfea9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.720121 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d14ce64-f501-4c56-aeff-137f931bfea9-crio-socket\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.720300 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.720357 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720329    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d14ce64-f501-4c56-aeff-137f931bfea9-data-volume\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.720412 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720371    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb95c197-005f-4deb-b563-107e7e0aff33-tmp-dir\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.720533 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720505    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d14ce64-f501-4c56-aeff-137f931bfea9-crio-socket\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.720622 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720604    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb95c197-005f-4deb-b563-107e7e0aff33-config-volume\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.720760 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.720737    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.723544 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.723512    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb95c197-005f-4deb-b563-107e7e0aff33-metrics-tls\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.723669 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.723521    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d14ce64-f501-4c56-aeff-137f931bfea9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.723669 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.723567    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ebd1a13-21bc-4de8-aaad-4e289981ee10-cert\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:51.744811 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.744778    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7dcp\" (UniqueName: \"kubernetes.io/projected/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-api-access-h7dcp\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.745238 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.745219    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpqk8\" (UniqueName: \"kubernetes.io/projected/fb95c197-005f-4deb-b563-107e7e0aff33-kube-api-access-jpqk8\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.745801 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.745783    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndnng\" (UniqueName: \"kubernetes.io/projected/6ebd1a13-21bc-4de8-aaad-4e289981ee10-kube-api-access-ndnng\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:51.779750 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.779706    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:51.783440 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.783419    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:38:51.805080 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:51.805050    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:38:52.090660 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:52.090515    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-drhmz"]
Oct 30 00:38:52.102726 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:52.102699    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qlw4m"]
Oct 30 00:38:52.105386 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:52.105355    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb95c197_005f_4deb_b563_107e7e0aff33.slice/crio-332f610b2772c7b3713325f1b5c74e11d9b9ae303f17c529b59144f5e12b55ed WatchSource:0}: Error finding container 332f610b2772c7b3713325f1b5c74e11d9b9ae303f17c529b59144f5e12b55ed: Status 404 returned error can't find the container with id 332f610b2772c7b3713325f1b5c74e11d9b9ae303f17c529b59144f5e12b55ed
Oct 30 00:38:52.117303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:52.117277    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-7zp52"]
Oct 30 00:38:52.121250 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:52.121217    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ebd1a13_21bc_4de8_aaad_4e289981ee10.slice/crio-aca378dc272d1049e9e21eaadf0d6d9cc658489b072275ba9f35c262000beae3 WatchSource:0}: Error finding container aca378dc272d1049e9e21eaadf0d6d9cc658489b072275ba9f35c262000beae3: Status 404 returned error can't find the container with id aca378dc272d1049e9e21eaadf0d6d9cc658489b072275ba9f35c262000beae3
Oct 30 00:38:52.360906 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:52.360806    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7zp52" event={"ID":"6ebd1a13-21bc-4de8-aaad-4e289981ee10","Type":"ContainerStarted","Data":"aca378dc272d1049e9e21eaadf0d6d9cc658489b072275ba9f35c262000beae3"}
Oct 30 00:38:52.362303 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:52.362267    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-drhmz" event={"ID":"0d14ce64-f501-4c56-aeff-137f931bfea9","Type":"ContainerStarted","Data":"a4e029fa80d9f479231578aa06916a47318e84af836f152dd1be05fb7cecd2af"}
Oct 30 00:38:52.362454 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:52.362334    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-drhmz" event={"ID":"0d14ce64-f501-4c56-aeff-137f931bfea9","Type":"ContainerStarted","Data":"d94b07be11a7323427254464954276c4ad0efdb139aef187ea01b04a44e71af0"}
Oct 30 00:38:52.363497 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:52.363466    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qlw4m" event={"ID":"fb95c197-005f-4deb-b563-107e7e0aff33","Type":"ContainerStarted","Data":"332f610b2772c7b3713325f1b5c74e11d9b9ae303f17c529b59144f5e12b55ed"}
Oct 30 00:38:53.093358 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.093267    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:38:53.093539 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.093271    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:38:53.093539 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.093436    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:38:53.110642 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.110023    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 00:38:53.110642 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.110065    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 00:38:53.122733 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.122698    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 00:38:53.122888 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.122725    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 00:38:53.122888 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.122816    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-9xms5\""
Oct 30 00:38:53.123003 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.122896    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-smcj2\""
Oct 30 00:38:53.123003 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.122960    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-44cn9\""
Oct 30 00:38:53.371421 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:53.371331    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-drhmz" event={"ID":"0d14ce64-f501-4c56-aeff-137f931bfea9","Type":"ContainerStarted","Data":"32ae4779286720a25bb4f08629add67021ca521ffb7113f0d76271cf5b994519"}
Oct 30 00:38:54.376343 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:54.376307    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-drhmz" event={"ID":"0d14ce64-f501-4c56-aeff-137f931bfea9","Type":"ContainerStarted","Data":"72f81be74535e5d1ce57dcfbf1346c0f57e5bc92b7785f2c2923711e879c87df"}
Oct 30 00:38:54.378518 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:54.378495    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qlw4m" event={"ID":"fb95c197-005f-4deb-b563-107e7e0aff33","Type":"ContainerStarted","Data":"505865bcb9aa24335b8c4d3cdf37dbdbee2cc0cb7c490eecda691c9932050cfd"}
Oct 30 00:38:54.378613 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:54.378543    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qlw4m" event={"ID":"fb95c197-005f-4deb-b563-107e7e0aff33","Type":"ContainerStarted","Data":"26b5b8a1dafc6ee934c4809de5610cea3fad3b52273790393e794ae51adfae56"}
Oct 30 00:38:54.378680 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:54.378664    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:38:54.424719 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:54.424650    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-drhmz" podStartSLOduration=1.382408363 podStartE2EDuration="3.424630769s" totalImagesPullingTime="2.042222406s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 00:38:51 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:54.423756525 +0000 UTC m=+48.788564151" watchObservedRunningTime="2026-10-30 00:38:54.424630769 +0000 UTC m=+48.789438398"
Oct 30 00:38:54.468913 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:54.468855    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-qlw4m" podStartSLOduration=1.6696457040000001 podStartE2EDuration="3.468836619s" totalImagesPullingTime="1.799190915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:54.468130372 +0000 UTC m=+48.832938024" watchObservedRunningTime="2026-10-30 00:38:54.468836619 +0000 UTC m=+48.833644246"
Oct 30 00:38:55.185988 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.185903    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"]
Oct 30 00:38:55.188191 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.188161    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:38:55.205802 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.205774    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 00:38:55.206082 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.206069    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xt47p\""
Oct 30 00:38:55.220610 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.220581    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"]
Oct 30 00:38:55.348860 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.348813    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e25b748-6ffe-488b-a210-24f1b4d1ce57-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-5tklp\" (UID: \"5e25b748-6ffe-488b-a210-24f1b4d1ce57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:38:55.382046 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.382010    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-7zp52" event={"ID":"6ebd1a13-21bc-4de8-aaad-4e289981ee10","Type":"ContainerStarted","Data":"9f530fe0c0c741c71b3ef97f6b7a6a7fc402f6013c36a964060d203b36f666e3"}
Oct 30 00:38:55.428799 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.428750    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-7zp52" podStartSLOduration=1.7235738729999999 podStartE2EDuration="4.428734428s" totalImagesPullingTime="2.705160555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:55.428278643 +0000 UTC m=+49.793086268" watchObservedRunningTime="2026-10-30 00:38:55.428734428 +0000 UTC m=+49.793542034"
Oct 30 00:38:55.450185 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.450091    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e25b748-6ffe-488b-a210-24f1b4d1ce57-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-5tklp\" (UID: \"5e25b748-6ffe-488b-a210-24f1b4d1ce57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:38:55.452827 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.452796    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e25b748-6ffe-488b-a210-24f1b4d1ce57-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-5tklp\" (UID: \"5e25b748-6ffe-488b-a210-24f1b4d1ce57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:38:55.501340 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.501292    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:38:55.797444 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.797421    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"]
Oct 30 00:38:55.799979 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:55.799948    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e25b748_6ffe_488b_a210_24f1b4d1ce57.slice/crio-a3fc8b43d3536f447460a589b34a6ba1cbe1c4da07574f8e4234d1fddb733ab9 WatchSource:0}: Error finding container a3fc8b43d3536f447460a589b34a6ba1cbe1c4da07574f8e4234d1fddb733ab9: Status 404 returned error can't find the container with id a3fc8b43d3536f447460a589b34a6ba1cbe1c4da07574f8e4234d1fddb733ab9
Oct 30 00:38:55.963419 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.963386    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-64684dbfbb-flsst"]
Oct 30 00:38:55.965136 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.965122    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:55.984498 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.984471    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 00:38:55.987109 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.987084    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 00:38:55.988388 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.988369    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64684dbfbb-flsst"]
Oct 30 00:38:55.989287 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.989265    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 00:38:55.989287 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.989279    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 00:38:55.989432 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.989315    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 00:38:55.989432 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.989355    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 00:38:55.989611 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:55.989543    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 00:38:56.083783 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.083699    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ptmsd\""
Oct 30 00:38:56.157445 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.157413    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.157445 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.157447    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.157645 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.157483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.157645 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.157527    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.157645 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.157613    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.157645 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.157637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwlfj\" (UniqueName: \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.157759 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.157655    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.259761 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.259037    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.259761 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.259083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.259761 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.259144    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.259761 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.259189    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwlfj\" (UniqueName: \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.259761 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.259217    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.259761 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.259268    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.259761 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.259294    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.260464 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.260434    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.260540 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.260450    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.260764 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.260735    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.260884 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.260795    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.265825 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.265763    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.266945 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.266902    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.280250 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.280224    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwlfj\" (UniqueName: \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.386597 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.386499    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp" event={"ID":"5e25b748-6ffe-488b-a210-24f1b4d1ce57","Type":"ContainerStarted","Data":"a3fc8b43d3536f447460a589b34a6ba1cbe1c4da07574f8e4234d1fddb733ab9"}
Oct 30 00:38:56.574756 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.574723    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:38:56.913963 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:56.913883    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64684dbfbb-flsst"]
Oct 30 00:38:56.917599 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:56.917567    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636aa0c9_51c4_43d1_ad08_78922681e9d7.slice/crio-207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf WatchSource:0}: Error finding container 207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf: Status 404 returned error can't find the container with id 207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf
Oct 30 00:38:57.390789 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:57.390752    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp" event={"ID":"5e25b748-6ffe-488b-a210-24f1b4d1ce57","Type":"ContainerStarted","Data":"4b7e9cb73bf9ebc2186a702c6133faa22118df6f056462fb858f5ebab0bc7653"}
Oct 30 00:38:57.391308 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:57.390901    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:38:57.392040 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:57.392014    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64684dbfbb-flsst" event={"ID":"636aa0c9-51c4-43d1-ad08-78922681e9d7","Type":"ContainerStarted","Data":"207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf"}
Oct 30 00:38:57.395779 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:57.395761    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:38:57.433165 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:57.433120    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp" podStartSLOduration=1.5411256070000001 podStartE2EDuration="2.433106909s" totalImagesPullingTime="891.981302ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:38:57.432287046 +0000 UTC m=+51.797094682" watchObservedRunningTime="2026-10-30 00:38:57.433106909 +0000 UTC m=+51.797914534"
Oct 30 00:38:58.393556 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.392420    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"]
Oct 30 00:38:58.394740 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.394707    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.428786 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.428760    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 00:38:58.428786 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.428779    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 00:38:58.433547 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.433520    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 00:38:58.433659 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.433563    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 00:38:58.440887 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.440847    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 00:38:58.441263 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.441241    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qwk9j\""
Oct 30 00:38:58.442257 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.442238    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"]
Oct 30 00:38:58.473977 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.473940    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.474121 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.474027    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.474121 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.474065    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.474121 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.474112    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clvh4\" (UniqueName: \"kubernetes.io/projected/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-kube-api-access-clvh4\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.575256 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.575211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clvh4\" (UniqueName: \"kubernetes.io/projected/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-kube-api-access-clvh4\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.575417 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.575298    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.575417 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.575375    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.575417 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.575402    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.576120 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.576092    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.578081 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.578061    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.578576 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.578554    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.599200 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.597619    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clvh4\" (UniqueName: \"kubernetes.io/projected/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-kube-api-access-clvh4\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:58.709295 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:58.709205    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:38:59.010575 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:59.010501    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"]
Oct 30 00:38:59.022195 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:38:59.022147    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f7acedd_ae54_4ad5_bef2_da836c8ef78e.slice/crio-4708b81382efc61bab8901a2614a38bdef1da5c138e4f6a804171929ff55c57e WatchSource:0}: Error finding container 4708b81382efc61bab8901a2614a38bdef1da5c138e4f6a804171929ff55c57e: Status 404 returned error can't find the container with id 4708b81382efc61bab8901a2614a38bdef1da5c138e4f6a804171929ff55c57e
Oct 30 00:38:59.400219 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:38:59.400157    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs" event={"ID":"1f7acedd-ae54-4ad5-bef2-da836c8ef78e","Type":"ContainerStarted","Data":"4708b81382efc61bab8901a2614a38bdef1da5c138e4f6a804171929ff55c57e"}
Oct 30 00:39:00.404868 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:00.404780    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64684dbfbb-flsst" event={"ID":"636aa0c9-51c4-43d1-ad08-78922681e9d7","Type":"ContainerStarted","Data":"c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59"}
Oct 30 00:39:00.406774 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:00.406748    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs" event={"ID":"1f7acedd-ae54-4ad5-bef2-da836c8ef78e","Type":"ContainerStarted","Data":"15d3510e22b285be995c3b859431a5a1b82569185085d96c03f38d57cfcb00ce"}
Oct 30 00:39:00.406880 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:00.406790    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs" event={"ID":"1f7acedd-ae54-4ad5-bef2-da836c8ef78e","Type":"ContainerStarted","Data":"349e5c312716255a56d624660fc6792a5e5a814207ebf16fbcc388ec4c641066"}
Oct 30 00:39:00.464900 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:00.464848    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-64684dbfbb-flsst" podStartSLOduration=2.809818695 podStartE2EDuration="5.464834515s" totalImagesPullingTime="2.65501582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:00.464340416 +0000 UTC m=+54.829148053" watchObservedRunningTime="2026-10-30 00:39:00.464834515 +0000 UTC m=+54.829642137"
Oct 30 00:39:00.505982 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:00.505917    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs" podStartSLOduration=1.494273965 podStartE2EDuration="2.505903087s" totalImagesPullingTime="1.011629122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:00.505564275 +0000 UTC m=+54.870371900" watchObservedRunningTime="2026-10-30 00:39:00.505903087 +0000 UTC m=+54.870710712"
Oct 30 00:39:01.327775 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:01.327748    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:02.974416 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:02.974378    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-84864"]
Oct 30 00:39:02.978360 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:02.978332    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.006551 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.006518    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5nmrp\""
Oct 30 00:39:03.006551 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.006524    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 00:39:03.006551 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.006543    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 00:39:03.011978 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.011957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f11449-4b9d-44df-8fed-68f31ded1601-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.012071 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.011990    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95rd7\" (UniqueName: \"kubernetes.io/projected/47f11449-4b9d-44df-8fed-68f31ded1601-kube-api-access-95rd7\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.012122 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.012075    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.012235 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.012218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.019007 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.018987    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-84864"]
Oct 30 00:39:03.080224 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.080189    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pztnw"]
Oct 30 00:39:03.082528 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.082504    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"]
Oct 30 00:39:03.082659 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.082647    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.084623 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.084602    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.106935 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.106911    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 00:39:03.107133 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.107002    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 00:39:03.112734 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.112699    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h74d\" (UniqueName: \"kubernetes.io/projected/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-kube-api-access-5h74d\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.112861 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.112750    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.112921 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.112900    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-root\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.112972 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.112940    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f11449-4b9d-44df-8fed-68f31ded1601-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.113075 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113056    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.113133 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-sys\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.113210 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113156    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95rd7\" (UniqueName: \"kubernetes.io/projected/47f11449-4b9d-44df-8fed-68f31ded1601-kube-api-access-95rd7\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.113348 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113334    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.113403 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113380    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.113458 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113427    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-metrics-client-ca\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.113525 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113496    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.113578 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113541    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.113643 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113618    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-tls\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.113695 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113663    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/162c7603-5367-4d79-8ff6-23b05743cf35-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.113751 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113698    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.113800 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113745    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.113800 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113762    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f11449-4b9d-44df-8fed-68f31ded1601-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.113800 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113783    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-textfile\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.113920 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113840    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qms8\" (UniqueName: \"kubernetes.io/projected/162c7603-5367-4d79-8ff6-23b05743cf35-kube-api-access-6qms8\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.113920 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.113905    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-wtmp\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.114902 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.114877    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 00:39:03.114902 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.114891    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p7bx4\""
Oct 30 00:39:03.115054 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.114966    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 00:39:03.115054 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.114966    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w7tqr\""
Oct 30 00:39:03.115054 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.115015    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 00:39:03.115054 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.115020    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 00:39:03.116138 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.116118    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.116399 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.116373    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"]
Oct 30 00:39:03.116669 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.116651    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.146250 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.146217    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95rd7\" (UniqueName: \"kubernetes.io/projected/47f11449-4b9d-44df-8fed-68f31ded1601-kube-api-access-95rd7\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.214650 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214608    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-textfile\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214650 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214646    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qms8\" (UniqueName: \"kubernetes.io/projected/162c7603-5367-4d79-8ff6-23b05743cf35-kube-api-access-6qms8\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.214650 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214662    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-wtmp\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h74d\" (UniqueName: \"kubernetes.io/projected/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-kube-api-access-5h74d\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214710    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214743    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-root\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214771    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214797    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-sys\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214841    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214850    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-root\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214871    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-metrics-client-ca\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.214955 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214944    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.215446 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215018    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-tls\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.215446 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215045    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-textfile\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.215446 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215050    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/162c7603-5367-4d79-8ff6-23b05743cf35-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.215446 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214947    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-sys\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.215446 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.214846    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-wtmp\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.215446 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215108    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.215446 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215169    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.215971 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215526    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/162c7603-5367-4d79-8ff6-23b05743cf35-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.215971 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215531    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-metrics-client-ca\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.215971 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.215955    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.216119 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.216090    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.216414 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.216283    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.217489 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.217464    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.217634 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.217612    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.217747 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.217731    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.217974 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.217958    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-tls\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.235379 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.235332    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qms8\" (UniqueName: \"kubernetes.io/projected/162c7603-5367-4d79-8ff6-23b05743cf35-kube-api-access-6qms8\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.235938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.235922    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h74d\" (UniqueName: \"kubernetes.io/projected/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-kube-api-access-5h74d\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.290934 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.290900    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:03.397563 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.397517    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:03.401878 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.401860    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:03.408205 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:03.408153    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5fbc6ed_d5e4_4b95_8e25_f0ffdc2861bc.slice/crio-250ca245080b48e3808e3d34afcb8a16f77cc0ba2da261f1d9a89a58d9493dd1 WatchSource:0}: Error finding container 250ca245080b48e3808e3d34afcb8a16f77cc0ba2da261f1d9a89a58d9493dd1: Status 404 returned error can't find the container with id 250ca245080b48e3808e3d34afcb8a16f77cc0ba2da261f1d9a89a58d9493dd1
Oct 30 00:39:03.416100 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.416073    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pztnw" event={"ID":"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc","Type":"ContainerStarted","Data":"250ca245080b48e3808e3d34afcb8a16f77cc0ba2da261f1d9a89a58d9493dd1"}
Oct 30 00:39:03.604854 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.604813    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-84864"]
Oct 30 00:39:03.609117 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:03.609088    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47f11449_4b9d_44df_8fed_68f31ded1601.slice/crio-aa5b97fc6fd53b4d0b34492332c304107b3aa0a60224b259b757ab9c3ef6e286 WatchSource:0}: Error finding container aa5b97fc6fd53b4d0b34492332c304107b3aa0a60224b259b757ab9c3ef6e286: Status 404 returned error can't find the container with id aa5b97fc6fd53b4d0b34492332c304107b3aa0a60224b259b757ab9c3ef6e286
Oct 30 00:39:03.695677 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:03.695654    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"]
Oct 30 00:39:03.707146 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:03.707123    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod162c7603_5367_4d79_8ff6_23b05743cf35.slice/crio-2970c24f126574f409849126cf31e82193b6f5d93e03621186de59951da5ae7b WatchSource:0}: Error finding container 2970c24f126574f409849126cf31e82193b6f5d93e03621186de59951da5ae7b: Status 404 returned error can't find the container with id 2970c24f126574f409849126cf31e82193b6f5d93e03621186de59951da5ae7b
Oct 30 00:39:04.384397 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.384372    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:04.412503 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.412457    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:39:04.415886 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.415866    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.428627 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.425126    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h" event={"ID":"162c7603-5367-4d79-8ff6-23b05743cf35","Type":"ContainerStarted","Data":"2970c24f126574f409849126cf31e82193b6f5d93e03621186de59951da5ae7b"}
Oct 30 00:39:04.428627 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.428249    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864" event={"ID":"47f11449-4b9d-44df-8fed-68f31ded1601","Type":"ContainerStarted","Data":"5bc91657c18c2da00b8e098741e1bea218f986d6656830a53fac765a74051457"}
Oct 30 00:39:04.428627 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.428315    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864" event={"ID":"47f11449-4b9d-44df-8fed-68f31ded1601","Type":"ContainerStarted","Data":"b24c358ee19758dc87fcd3d17e8d723ff0a0ec58cfa8bbacaa788b997cec72f0"}
Oct 30 00:39:04.428627 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.428350    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864" event={"ID":"47f11449-4b9d-44df-8fed-68f31ded1601","Type":"ContainerStarted","Data":"aa5b97fc6fd53b4d0b34492332c304107b3aa0a60224b259b757ab9c3ef6e286"}
Oct 30 00:39:04.448208 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.446998    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 00:39:04.448208 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.447207    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 00:39:04.449370 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.449352    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 00:39:04.449559 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.449544    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 00:39:04.449704 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.449691    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 00:39:04.449854 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.449842    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 00:39:04.450201 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.450167    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 00:39:04.450374 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.450361    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 00:39:04.456374 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.455929    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lpt8z\""
Oct 30 00:39:04.456374 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.455981    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 00:39:04.458272 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.458254    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:39:04.526492 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526405    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.526492 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526447    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.526728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526580    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.526728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526622    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.526728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526638    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.526728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526662    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.526728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526679    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.526728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526714    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fn4jb\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.527020 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526749    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.527020 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.527020 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526806    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.527020 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.527020 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.526857    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.627852 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.627812    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628029 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.627864    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628029 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.627915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628029 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.627973    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628029 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628001    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628278 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628052    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628278 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628093    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628278 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628278 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628153    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628278 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628198    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628278 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628223    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn4jb\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628278 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628262    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.628554 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.628310    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.630400 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.630369    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.630631 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.630602    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.630710 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.630643    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.632391 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.632309    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.632873 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.632848    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.633487 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.633462    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.634061 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.634036    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.634061 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.634052    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.634775 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.634349    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.634775 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.634735    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.634922 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.634792    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.635281 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.635255    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.662160 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.662133    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn4jb\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:04.765909 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:04.765880    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:05.099027 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.098995    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:39:05.102969 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:05.102933    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9bfb9c_bc15_48d5_98a3_5bd1761fde16.slice/crio-f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117 WatchSource:0}: Error finding container f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117: Status 404 returned error can't find the container with id f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117
Oct 30 00:39:05.216438 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.216407    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"]
Oct 30 00:39:05.220648 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.220622    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.236400 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.236377    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 00:39:05.236533 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.236469    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hs87l\""
Oct 30 00:39:05.236533 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.236496    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 00:39:05.236714 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.236700    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 00:39:05.236852 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.236836    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 00:39:05.236914 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.236871    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 00:39:05.237003 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.236989    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-907sh4mbvn436\""
Oct 30 00:39:05.248816 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.248786    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"]
Oct 30 00:39:05.338911 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.338878    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zhgq\" (UniqueName: \"kubernetes.io/projected/4d53587f-0770-4a39-a369-dcc257c949f5-kube-api-access-4zhgq\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339086 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.338927    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-grpc-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339086 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.339050    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53587f-0770-4a39-a369-dcc257c949f5-tmp\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339158 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.339099    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d53587f-0770-4a39-a369-dcc257c949f5-metrics-client-ca\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339226 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.339153    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339226 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.339198    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339287 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.339231    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339287 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.339251    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.339287 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.339271    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.436667 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.436632    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h" event={"ID":"162c7603-5367-4d79-8ff6-23b05743cf35","Type":"ContainerStarted","Data":"fdfb3bb83093a0ee5f68d1968c1ed10bb1ded226d436b19bd353bab2c5dc30e1"}
Oct 30 00:39:05.437114 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.436685    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h" event={"ID":"162c7603-5367-4d79-8ff6-23b05743cf35","Type":"ContainerStarted","Data":"2ce7ec80776e9965fe86aa0c5064aa06181bbaef4e23890ab70451392c081b6b"}
Oct 30 00:39:05.437114 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.436710    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h" event={"ID":"162c7603-5367-4d79-8ff6-23b05743cf35","Type":"ContainerStarted","Data":"71c9c4772b3622bcd12fba1475350943c1601149354ef64f4d0c39048fc4d451"}
Oct 30 00:39:05.438372 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.438349    2571 generic.go:356] "Generic (PLEG): container finished" podID="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" containerID="4adccfc08ebac297ac449ffdc1a197af8a29a197544082196a7b25b754ae4c0b" exitCode=0
Oct 30 00:39:05.438476 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.438416    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pztnw" event={"ID":"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc","Type":"ContainerDied","Data":"4adccfc08ebac297ac449ffdc1a197af8a29a197544082196a7b25b754ae4c0b"}
Oct 30 00:39:05.439682 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439658    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53587f-0770-4a39-a369-dcc257c949f5-tmp\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.439792 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439692    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerStarted","Data":"f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117"}
Oct 30 00:39:05.439792 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439698    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d53587f-0770-4a39-a369-dcc257c949f5-metrics-client-ca\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.439792 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.439943 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439845    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.439943 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439884    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.439943 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439914    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.440079 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.439973    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.440079 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.440042    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zhgq\" (UniqueName: \"kubernetes.io/projected/4d53587f-0770-4a39-a369-dcc257c949f5-kube-api-access-4zhgq\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.440198 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.440096    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53587f-0770-4a39-a369-dcc257c949f5-tmp\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.440198 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.440108    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-grpc-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.440947 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.440911    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d53587f-0770-4a39-a369-dcc257c949f5-metrics-client-ca\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.444496 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.443692    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.444496 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.443837    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-grpc-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.444496 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.444245    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.444885 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.444860    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.445114 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.445091    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.449812 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.449710    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864" event={"ID":"47f11449-4b9d-44df-8fed-68f31ded1601","Type":"ContainerStarted","Data":"920d6a2370da7b78dab5bfd98291716b8fd9894201225aeee1792df5a19c1fad"}
Oct 30 00:39:05.450122 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.450099    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.459625 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.459600    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zhgq\" (UniqueName: \"kubernetes.io/projected/4d53587f-0770-4a39-a369-dcc257c949f5-kube-api-access-4zhgq\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.478119 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.478079    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h" podStartSLOduration=1.18413998 podStartE2EDuration="2.478068892s" totalImagesPullingTime="1.293928912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:05.477333628 +0000 UTC m=+59.842141264" watchObservedRunningTime="2026-10-30 00:39:05.478068892 +0000 UTC m=+59.842876516"
Oct 30 00:39:05.517232 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.517144    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864" podStartSLOduration=2.522868504 podStartE2EDuration="3.517125101s" totalImagesPullingTime="994.256597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:39:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:05.516037056 +0000 UTC m=+59.880844679" watchObservedRunningTime="2026-10-30 00:39:05.517125101 +0000 UTC m=+59.881932728"
Oct 30 00:39:05.547165 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.547140    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:05.860654 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:05.860622    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"]
Oct 30 00:39:05.863202 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:05.863141    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d53587f_0770_4a39_a369_dcc257c949f5.slice/crio-8bc0f39b14cac44d07cb4c31e4875b76787072021f455fdeed3132d74efb6382 WatchSource:0}: Error finding container 8bc0f39b14cac44d07cb4c31e4875b76787072021f455fdeed3132d74efb6382: Status 404 returned error can't find the container with id 8bc0f39b14cac44d07cb4c31e4875b76787072021f455fdeed3132d74efb6382
Oct 30 00:39:06.157934 ip-10-0-131-254 kubenswrapper[2571]: E1030 00:39:06.157895    2571 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9bfb9c_bc15_48d5_98a3_5bd1761fde16.slice/crio-conmon-4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a9bfb9c_bc15_48d5_98a3_5bd1761fde16.slice/crio-4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 00:39:06.456334 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.456255    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pztnw" event={"ID":"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc","Type":"ContainerStarted","Data":"2d40dd700b5c54d1b7fe3e7814c354711b1a576593f42d8354c184e2c60bfa25"}
Oct 30 00:39:06.456753 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.456353    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pztnw" event={"ID":"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc","Type":"ContainerStarted","Data":"32b920f0e23ff587dfd9f6efda6d18c94585152d5ec74cd779828a10da83adc9"}
Oct 30 00:39:06.458429 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.458293    2571 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f" exitCode=0
Oct 30 00:39:06.458429 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.458367    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f"}
Oct 30 00:39:06.460240 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.460210    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" event={"ID":"4d53587f-0770-4a39-a369-dcc257c949f5","Type":"ContainerStarted","Data":"8bc0f39b14cac44d07cb4c31e4875b76787072021f455fdeed3132d74efb6382"}
Oct 30 00:39:06.503837 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.503775    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pztnw" podStartSLOduration=2.435909191 podStartE2EDuration="3.503758785s" totalImagesPullingTime="973.890239ms" totalInitContainerRuntime="93.959355ms" isStatefulPod=true podCreationTimestamp="2026-10-30 00:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:06.503359926 +0000 UTC m=+60.868167542" watchObservedRunningTime="2026-10-30 00:39:06.503758785 +0000 UTC m=+60.868566411"
Oct 30 00:39:06.575669 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.575637    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:06.575832 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.575740    2571 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:06.581793 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:06.581765    2571 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:07.266249 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.266214    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-59ff97898d-4tlzq"]
Oct 30 00:39:07.269649 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.269626    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.287771 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.287745    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59ff97898d-4tlzq"]
Oct 30 00:39:07.323751 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.323716    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 00:39:07.358522 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.358522 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358483    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmtz7\" (UniqueName: \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.358758 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358619    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.358758 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358665    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.358862 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358763    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.358862 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.358949 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358924    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.358979 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.358957    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460482 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460429    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460497    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460534    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460564    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmtz7\" (UniqueName: \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460664    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460735    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460779    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.460938 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.460803    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.461906 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.461881    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.462691 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.462669    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.463243 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.463216    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.464095 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.464070    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.464449 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.464423    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.465111 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.465085    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.467075 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.467015    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.471900 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.471879    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:07.481363 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.481341    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmtz7\" (UniqueName: \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.583197 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.583148    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:07.644019 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.644000    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"]
Oct 30 00:39:07.647941 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.647236    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.664716 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.664688    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"]
Oct 30 00:39:07.665850 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.665829    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 00:39:07.670157 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.665982    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 00:39:07.670157 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.666211    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 00:39:07.670157 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.666386    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-crwd9\""
Oct 30 00:39:07.670157 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.666402    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 00:39:07.670157 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.666688    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-16hl7253o93j0\""
Oct 30 00:39:07.766104 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.766071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-client-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.766229 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.766190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.766298 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.766265    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-client-certs\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.766352 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.766319    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-tls\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.766352 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.766345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01d89188-b530-4373-9ea9-b58a4122c8cd-audit-log\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.766427 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.766397    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-metrics-server-audit-profiles\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.766472 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.766423    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct2s6\" (UniqueName: \"kubernetes.io/projected/01d89188-b530-4373-9ea9-b58a4122c8cd-kube-api-access-ct2s6\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867224 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867194    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-client-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867346 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867241    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867346 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867286    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-client-certs\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867346 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867322    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-tls\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867440 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867348    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01d89188-b530-4373-9ea9-b58a4122c8cd-audit-log\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867440 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867397    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-metrics-server-audit-profiles\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867440 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867421    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct2s6\" (UniqueName: \"kubernetes.io/projected/01d89188-b530-4373-9ea9-b58a4122c8cd-kube-api-access-ct2s6\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.867800 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867773    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01d89188-b530-4373-9ea9-b58a4122c8cd-audit-log\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.868044 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.867995    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.868523 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.868502    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-metrics-server-audit-profiles\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.870481 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.870455    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-client-certs\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.870546 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.870507    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-client-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.870628 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.870541    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-tls\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.891110 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.891081    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59ff97898d-4tlzq"]
Oct 30 00:39:07.893029 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.893004    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct2s6\" (UniqueName: \"kubernetes.io/projected/01d89188-b530-4373-9ea9-b58a4122c8cd-kube-api-access-ct2s6\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:07.893603 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:07.893575    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24c36423_7a10_4a80_a3c8_2e3835c3d12a.slice/crio-6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e WatchSource:0}: Error finding container 6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e: Status 404 returned error can't find the container with id 6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e
Oct 30 00:39:07.971620 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:07.971582    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:08.284511 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.284397    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"]
Oct 30 00:39:08.288473 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:08.288370    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d89188_b530_4373_9ea9_b58a4122c8cd.slice/crio-e0e0d4c950ce469ddd0e02a2c7a0f064ad98d31f34da87edb90b76ad7cdd9865 WatchSource:0}: Error finding container e0e0d4c950ce469ddd0e02a2c7a0f064ad98d31f34da87edb90b76ad7cdd9865: Status 404 returned error can't find the container with id e0e0d4c950ce469ddd0e02a2c7a0f064ad98d31f34da87edb90b76ad7cdd9865
Oct 30 00:39:08.345429 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.345399    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6959d644bc-xfsth"]
Oct 30 00:39:08.351395 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.351374    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.369447 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.369386    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 00:39:08.369447 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.369392    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 00:39:08.369959 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.369810    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 00:39:08.369959 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.369865    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 00:39:08.369959 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.369874    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 00:39:08.369959 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.369930    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-m2pgd\""
Oct 30 00:39:08.381295 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.381230    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 00:39:08.382850 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.382779    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6959d644bc-xfsth"]
Oct 30 00:39:08.471672 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.471644    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98" event={"ID":"01d89188-b530-4373-9ea9-b58a4122c8cd","Type":"ContainerStarted","Data":"e0e0d4c950ce469ddd0e02a2c7a0f064ad98d31f34da87edb90b76ad7cdd9865"}
Oct 30 00:39:08.473428 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473406    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.473522 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473444    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.473629 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473610    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.473664 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473640    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5497b\" (UniqueName: \"kubernetes.io/projected/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-kube-api-access-5497b\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.473728 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473715    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-federate-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.473781 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473764    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-serving-certs-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.473834 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.473903 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.473882    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-metrics-client-ca\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.474313 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.474292    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerStarted","Data":"798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe"}
Oct 30 00:39:08.474377 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.474362    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerStarted","Data":"4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca"}
Oct 30 00:39:08.474414 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.474407    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerStarted","Data":"1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e"}
Oct 30 00:39:08.476504 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.476386    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" event={"ID":"4d53587f-0770-4a39-a369-dcc257c949f5","Type":"ContainerStarted","Data":"f82855f471c0dba445be7fe27511c188e3e68a97ef4e105d923522c36e48a02c"}
Oct 30 00:39:08.476504 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.476437    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" event={"ID":"4d53587f-0770-4a39-a369-dcc257c949f5","Type":"ContainerStarted","Data":"990114e9298b7783d6c21e58d832a0c65009f36bee1b2e712970d1e487a96a86"}
Oct 30 00:39:08.476504 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.476460    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" event={"ID":"4d53587f-0770-4a39-a369-dcc257c949f5","Type":"ContainerStarted","Data":"1bcc060b0234c6133cce52ff05f099152b1e9f91f0b919c8901c994c6d358503"}
Oct 30 00:39:08.479273 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.479243    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff97898d-4tlzq" event={"ID":"24c36423-7a10-4a80-a3c8-2e3835c3d12a","Type":"ContainerStarted","Data":"3b999128fdbb7f9fe304379e317093d6be3fcbab8c5ba5df4074a529b290498b"}
Oct 30 00:39:08.479373 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.479299    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff97898d-4tlzq" event={"ID":"24c36423-7a10-4a80-a3c8-2e3835c3d12a","Type":"ContainerStarted","Data":"6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e"}
Oct 30 00:39:08.524835 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.524787    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-59ff97898d-4tlzq" podStartSLOduration=1.524768645 podStartE2EDuration="1.524768645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:08.523710105 +0000 UTC m=+62.888517748" watchObservedRunningTime="2026-10-30 00:39:08.524768645 +0000 UTC m=+62.889576270"
Oct 30 00:39:08.574980 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.574949    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.575104 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.575011    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.575162 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.575142    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.575224 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.575161    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5497b\" (UniqueName: \"kubernetes.io/projected/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-kube-api-access-5497b\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.575275 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.575245    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-federate-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.575341 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.575327    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-serving-certs-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.575375 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.575349    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.575547 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.575436    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-metrics-client-ca\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.576840 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.576806    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.577627 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.577552    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-metrics-client-ca\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.578694 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.578145    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-serving-certs-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.578694 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.578560    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.579698 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.579273    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.579698 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.579538    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.581637 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.581613    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-federate-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.601658 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.601626    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5497b\" (UniqueName: \"kubernetes.io/projected/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-kube-api-access-5497b\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:08.670656 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:08.670619    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:09.006312 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.006226    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6959d644bc-xfsth"]
Oct 30 00:39:09.021243 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:09.021211    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0fed46_f827_4a14_879c_b0c4a4fa7db8.slice/crio-eede4fd62fdeecb71c517704956e1c4ad7154c9df215916ed87b10bc3f78b642 WatchSource:0}: Error finding container eede4fd62fdeecb71c517704956e1c4ad7154c9df215916ed87b10bc3f78b642: Status 404 returned error can't find the container with id eede4fd62fdeecb71c517704956e1c4ad7154c9df215916ed87b10bc3f78b642
Oct 30 00:39:09.492670 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.491373    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth" event={"ID":"ae0fed46-f827-4a14-879c-b0c4a4fa7db8","Type":"ContainerStarted","Data":"eede4fd62fdeecb71c517704956e1c4ad7154c9df215916ed87b10bc3f78b642"}
Oct 30 00:39:09.505956 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.505914    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerStarted","Data":"1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8"}
Oct 30 00:39:09.506098 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.505990    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerStarted","Data":"32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062"}
Oct 30 00:39:09.506098 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.506028    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerStarted","Data":"be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8"}
Oct 30 00:39:09.511320 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.510756    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" event={"ID":"4d53587f-0770-4a39-a369-dcc257c949f5","Type":"ContainerStarted","Data":"3f280b125cc1ebc187718a526fd3b9dae14e2cbf93e1b266bc5edb6ff13160a9"}
Oct 30 00:39:09.511320 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.510820    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" event={"ID":"4d53587f-0770-4a39-a369-dcc257c949f5","Type":"ContainerStarted","Data":"6dcc424e60664ad655df7f6a87e05244025b7431cc99c9b20a55eae4341e366b"}
Oct 30 00:39:09.511320 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.510862    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" event={"ID":"4d53587f-0770-4a39-a369-dcc257c949f5","Type":"ContainerStarted","Data":"cfdf84164bc3804997a1b099c05a444cc8f2c3abb2b9a03f616c1804920f8538"}
Oct 30 00:39:09.575538 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:09.575483    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.410631935 podStartE2EDuration="5.575465932s" totalImagesPullingTime="3.148601058s" totalInitContainerRuntime="16.232939ms" isStatefulPod=false podCreationTimestamp="2026-10-30 00:39:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:09.573801696 +0000 UTC m=+63.938609323" watchObservedRunningTime="2026-10-30 00:39:09.575465932 +0000 UTC m=+63.940273557"
Oct 30 00:39:10.518407 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:10.516964    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98" event={"ID":"01d89188-b530-4373-9ea9-b58a4122c8cd","Type":"ContainerStarted","Data":"092e1a14f6705a8642a8ea126118c1541e5f132726da2a9a5a360d1f5f2ecdb2"}
Oct 30 00:39:10.518407 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:10.517954    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:10.557677 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:10.557624    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98" podStartSLOduration=2.232282759 podStartE2EDuration="3.557610029s" totalImagesPullingTime="1.32532727s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:10.556949264 +0000 UTC m=+64.921756886" watchObservedRunningTime="2026-10-30 00:39:10.557610029 +0000 UTC m=+64.922417654"
Oct 30 00:39:10.558185 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:10.558141    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv" podStartSLOduration=2.942516786 podStartE2EDuration="5.558132747s" totalImagesPullingTime="2.615615961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:39:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:09.627030389 +0000 UTC m=+63.991838027" watchObservedRunningTime="2026-10-30 00:39:10.558132747 +0000 UTC m=+64.922940369"
Oct 30 00:39:11.522557 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.522511    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth" event={"ID":"ae0fed46-f827-4a14-879c-b0c4a4fa7db8","Type":"ContainerStarted","Data":"1c7f1aac103b75fec28b4cfb72aa11a4b4611e5a251a2e0b30da5920798cf6a5"}
Oct 30 00:39:11.523088 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.522603    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth" event={"ID":"ae0fed46-f827-4a14-879c-b0c4a4fa7db8","Type":"ContainerStarted","Data":"77cbcb40eecbedcc472c063043acebe51e4997b2967182ec32c6c07deab37c55"}
Oct 30 00:39:11.523088 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.522636    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth" event={"ID":"ae0fed46-f827-4a14-879c-b0c4a4fa7db8","Type":"ContainerStarted","Data":"cff6c9ff943a04dfff7e2e6a77e9c3c3b2e16e56d5df9306c7ab5d91b67625ee"}
Oct 30 00:39:11.592444 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.592397    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth" podStartSLOduration=2.049668645 podStartE2EDuration="3.592384623s" totalImagesPullingTime="1.542715978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:39:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:11.59211307 +0000 UTC m=+65.956920694" watchObservedRunningTime="2026-10-30 00:39:11.592384623 +0000 UTC m=+65.957192272"
Oct 30 00:39:11.916916 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.916882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:39:11.916916 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.916935    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:11.930833 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.930805    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 00:39:11.930945 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.930882    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 00:39:11.940533 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.940504    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:39:11.940636 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:11.940511    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:12.018089 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.018051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:12.024462 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.024437    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-44cn9\""
Oct 30 00:39:12.031154 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.031122    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:39:12.031385 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.031368    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-9xms5\""
Oct 30 00:39:12.031477 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.031399    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 00:39:12.038387 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.038367    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:12.052078 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.052059    2571 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 00:39:12.061747 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.061721    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:12.327101 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.327060    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-52dwv"]
Oct 30 00:39:12.331080 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:12.331048    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1157b8_e332_4163_b20a_1edbbe85f786.slice/crio-aa7481591b6759b61b735054248a39b3f74d0b04eed2963b1f8fdfb4b13c9e5d WatchSource:0}: Error finding container aa7481591b6759b61b735054248a39b3f74d0b04eed2963b1f8fdfb4b13c9e5d: Status 404 returned error can't find the container with id aa7481591b6759b61b735054248a39b3f74d0b04eed2963b1f8fdfb4b13c9e5d
Oct 30 00:39:12.338617 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.338516    2571 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-smcj2\""
Oct 30 00:39:12.338682 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.338634    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x2s8q"]
Oct 30 00:39:12.340499 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:12.340477    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc64e9e6d_c2b7_4520_b5d9_e8eb86720bab.slice/crio-36918762d7bdd40bd5bcdc2e4d93c8a059f2e794161144f98f7f1b44c59213f5 WatchSource:0}: Error finding container 36918762d7bdd40bd5bcdc2e4d93c8a059f2e794161144f98f7f1b44c59213f5: Status 404 returned error can't find the container with id 36918762d7bdd40bd5bcdc2e4d93c8a059f2e794161144f98f7f1b44c59213f5
Oct 30 00:39:12.345844 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.345825    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:12.527529 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.527486    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x2s8q" event={"ID":"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab","Type":"ContainerStarted","Data":"36918762d7bdd40bd5bcdc2e4d93c8a059f2e794161144f98f7f1b44c59213f5"}
Oct 30 00:39:12.528684 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.528662    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-52dwv" event={"ID":"3a1157b8-e332-4163-b20a-1edbbe85f786","Type":"ContainerStarted","Data":"aa7481591b6759b61b735054248a39b3f74d0b04eed2963b1f8fdfb4b13c9e5d"}
Oct 30 00:39:12.643104 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:12.643008    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pgl9q"]
Oct 30 00:39:12.645502 ip-10-0-131-254 kubenswrapper[2571]: W1030 00:39:12.645477    2571 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd83da39_c295_436b_a07c_4c41c68de6be.slice/crio-5fd88bbcea9ab74151f52abd18f825194212aa80873e44b6c7d9971d3a86dad5 WatchSource:0}: Error finding container 5fd88bbcea9ab74151f52abd18f825194212aa80873e44b6c7d9971d3a86dad5: Status 404 returned error can't find the container with id 5fd88bbcea9ab74151f52abd18f825194212aa80873e44b6c7d9971d3a86dad5
Oct 30 00:39:13.538562 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:13.538528    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pgl9q" event={"ID":"cd83da39-c295-436b-a07c-4c41c68de6be","Type":"ContainerStarted","Data":"5fd88bbcea9ab74151f52abd18f825194212aa80873e44b6c7d9971d3a86dad5"}
Oct 30 00:39:13.546748 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:13.546714    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-52dwv" event={"ID":"3a1157b8-e332-4163-b20a-1edbbe85f786","Type":"ContainerStarted","Data":"6a4712cae33ce82815cff70f760cb6eadc1c271c77616179e6043999d9183551"}
Oct 30 00:39:14.553520 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:14.553484    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-52dwv" event={"ID":"3a1157b8-e332-4163-b20a-1edbbe85f786","Type":"ContainerStarted","Data":"5a29326a48f239f6ef26e63776da1328df25241d4929823163c936a89c1a4fb9"}
Oct 30 00:39:14.594601 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:14.594484    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-52dwv" podStartSLOduration=67.599911638 podStartE2EDuration="1m8.594447031s" totalImagesPullingTime="994.535393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:14.592524422 +0000 UTC m=+68.957332068" watchObservedRunningTime="2026-10-30 00:39:14.594447031 +0000 UTC m=+68.959254660"
Oct 30 00:39:16.530702 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:16.530619    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:16.565304 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:16.565273    2571 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pgl9q" event={"ID":"cd83da39-c295-436b-a07c-4c41c68de6be","Type":"ContainerStarted","Data":"c63faab59a3c3dae8d9c94f0169f9c8e41e3f1acaded178a97feb549187bc8b5"}
Oct 30 00:39:16.565492 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:16.565335    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:16.630713 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:16.630640    2571 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pgl9q" podStartSLOduration=67.015022253 podStartE2EDuration="1m10.630619925s" totalImagesPullingTime="3.615597672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:38:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:39:16.627608901 +0000 UTC m=+70.992416543" watchObservedRunningTime="2026-10-30 00:39:16.630619925 +0000 UTC m=+70.995427556"
Oct 30 00:39:17.295913 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.295880    2571 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59ff97898d-4tlzq"]
Oct 30 00:39:17.320759 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.320729    2571 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-78cf5d5fdb-fxvv5"]
Oct 30 00:39:17.324405 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.324381    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.342106 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.342080    2571 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78cf5d5fdb-fxvv5"]
Oct 30 00:39:17.471620 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471589    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.471620 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471625    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.471794 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471646    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.471794 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471724    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.471851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471812    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.471851 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471843    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.471943 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471930    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.471976 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.471960    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8229s\" (UniqueName: \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573210 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573167    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573566 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573226    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8229s\" (UniqueName: \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573566 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573258    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573566 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573304    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573566 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573752 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573609    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573752 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573656    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.573752 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.573691    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.574158 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.574135    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.574291 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.574275    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.574346 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.574293    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.574682 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.574662    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.574777 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.574753    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.575795 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.575774    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.575978 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.575959    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.583276 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.583252    2571 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:17.600404 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.600382    2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8229s\" (UniqueName: \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.638810 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.638784    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:17.840164 ip-10-0-131-254 kubenswrapper[2571]: I1030 00:39:17.840026    2571 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:39:17.840235 ip-10-0-131-254 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 00:39:17.847936 ip-10-0-131-254 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 00:39:17.848187 ip-10-0-131-254 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 00:39:17.848245 ip-10-0-131-254 systemd[1]: kubelet.service: Consumed 4.690s CPU time.
Oct 30 00:39:17.856862 ip-10-0-131-254 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 00:39:17.928576 ip-10-0-131-254 kubenswrapper[6011]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:39:17.928576 ip-10-0-131-254 kubenswrapper[6011]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 00:39:17.928576 ip-10-0-131-254 kubenswrapper[6011]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:39:17.928576 ip-10-0-131-254 kubenswrapper[6011]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 00:39:17.930825 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930741    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.930825 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930826    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930830    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930834    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930836    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930839    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930842    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930845    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930848    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930850    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930853    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930855    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930859    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930861    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930864    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930866    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930869    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930872    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930874    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930876    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930879    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.930906 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930888    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930891    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930893    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930896    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930898    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930901    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930904    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930906    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930909    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930911    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930915    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930917    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930920    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930923    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930927    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930932    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930935    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930937    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930940    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.931416 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930942    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930945    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930947    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930950    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930952    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930955    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930957    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930959    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930962    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930964    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930967    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930970    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930972    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930975    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930978    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930981    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930984    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930986    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930989    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930992    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.931862 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930994    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930997    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.930999    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931002    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931004    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931006    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931009    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931011    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931014    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931018    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931022    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931025    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931029    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931032    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931034    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931037    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931039    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931042    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931044    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.932351 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931047    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931049    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931052    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931054    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931057    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931059    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931062    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931064    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931068    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931071    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931074    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931077    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931079    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931081    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931084    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931096    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931099    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931102    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931104    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931107    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.932801 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931109    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931112    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931114    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931117    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931119    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931122    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.931125    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932101    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932111    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932114    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932117    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932120    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932122    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932126    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932128    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932131    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932134    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932136    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932139    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.933275 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932141    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932144    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932147    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932149    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932152    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932155    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932157    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932160    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932164    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932168    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932190    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932197    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932203    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932209    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932214    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932219    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932222    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932225    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932227    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.933708 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932230    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932233    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932236    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932239    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932242    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932245    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932247    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932250    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932252    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932255    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932257    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932260    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932262    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932265    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932268    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932271    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932274    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932280    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932283    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.934155 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932285    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932288    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932290    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932293    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932295    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932298    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932301    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932303    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932305    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932308    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932310    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932313    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932316    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932318    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932321    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932324    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932326    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932329    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932331    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.934635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932335    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932338    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932340    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932343    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932346    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932348    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932351    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932353    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932357    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932359    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932362    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932365    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932368    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932371    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932374    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932376    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932379    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932382    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932384    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932387    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.935080 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932389    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932392    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932394    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932397    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932399    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932402    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932404    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932407    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932409    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932412    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932415    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932417    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932420    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932422    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932426    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932429    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932432    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932938    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932944    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932947    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.935635 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932949    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932952    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932955    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932959    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932962    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932964    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932967    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932969    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932972    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932974    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932977    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932981    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932985    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932988    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932991    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932994    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932997    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.932999    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933002    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.936099 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933004    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933007    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933015    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933019    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933023    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933026    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933028    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933032    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933034    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933037    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933040    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933042    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933045    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933047    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933050    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933052    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933056    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933059    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933061    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.936565 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933064    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933066    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933069    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933073    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933075    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933078    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933080    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933083    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933085    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933088    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933090    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933092    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933095    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933097    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933100    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933102    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933106    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933109    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933112    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933114    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.937058 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933117    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933120    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933122    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933125    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933127    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933129    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933132    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933135    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933137    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933141    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933144    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933146    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933149    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933152    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933154    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933157    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933159    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933162    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933164    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.937597 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933167    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933185    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933190    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933195    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933200    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933204    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933209    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933213    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933217    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933220    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933223    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933226    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933229    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933232    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933234    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933237    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933239    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933242    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933244    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933247    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.938147 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933250    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933253    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933257    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933259    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933262    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.933264    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933374    6011 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933390    6011 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933399    6011 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933404    6011 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933408    6011 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933412    6011 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933416    6011 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933420    6011 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933423    6011 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933426    6011 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933430    6011 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933434    6011 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933437    6011 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933440    6011 flags.go:64] FLAG: --cgroup-root=""
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933443    6011 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933446    6011 flags.go:64] FLAG: --client-ca-file=""
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933449    6011 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933458    6011 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 00:39:17.938634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933465    6011 flags.go:64] FLAG: --cluster-domain=""
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933469    6011 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933472    6011 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933475    6011 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933479    6011 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933483    6011 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933487    6011 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933490    6011 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933493    6011 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933497    6011 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933500    6011 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933504    6011 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933507    6011 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933510    6011 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933514    6011 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933517    6011 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933520    6011 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933523    6011 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933526    6011 flags.go:64] FLAG: --enable-server="true"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933529    6011 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933536    6011 flags.go:64] FLAG: --event-burst="100"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933539    6011 flags.go:64] FLAG: --event-qps="50"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933542    6011 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933546    6011 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 00:39:17.939215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933549    6011 flags.go:64] FLAG: --eviction-hard=""
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933553    6011 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933556    6011 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933559    6011 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933562    6011 flags.go:64] FLAG: --eviction-soft=""
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933565    6011 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933568    6011 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933571    6011 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933579    6011 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933584    6011 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933587    6011 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933591    6011 flags.go:64] FLAG: --feature-gates=""
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933595    6011 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933598    6011 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933602    6011 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933605    6011 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933608    6011 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933611    6011 flags.go:64] FLAG: --help="false"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933614    6011 flags.go:64] FLAG: --hostname-override="ip-10-0-131-254.ec2.internal"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933618    6011 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933621    6011 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933624    6011 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933627    6011 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 00:39:17.939782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933631    6011 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933634    6011 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933637    6011 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933640    6011 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933649    6011 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933653    6011 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933656    6011 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933659    6011 flags.go:64] FLAG: --kube-reserved=""
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933662    6011 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933665    6011 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933669    6011 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933671    6011 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933674    6011 flags.go:64] FLAG: --lock-file=""
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933677    6011 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933680    6011 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933683    6011 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933693    6011 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933696    6011 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933708    6011 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933711    6011 flags.go:64] FLAG: --logging-format="text"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933715    6011 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933719    6011 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933722    6011 flags.go:64] FLAG: --manifest-url=""
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933725    6011 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933729    6011 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 00:39:17.940406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933733    6011 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933737    6011 flags.go:64] FLAG: --max-pods="110"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933740    6011 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933743    6011 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933746    6011 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933749    6011 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933752    6011 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933755    6011 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933758    6011 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933765    6011 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933771    6011 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933775    6011 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933778    6011 flags.go:64] FLAG: --pod-cidr=""
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933780    6011 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933784    6011 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933787    6011 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933790    6011 flags.go:64] FLAG: --port="10250"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933793    6011 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933795    6011 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-09906a28b26f72a39"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933798    6011 flags.go:64] FLAG: --qos-reserved=""
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933802    6011 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933805    6011 flags.go:64] FLAG: --register-node="true"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933808    6011 flags.go:64] FLAG: --register-with-taints=""
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933812    6011 flags.go:64] FLAG: --registry-burst="10"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933814    6011 flags.go:64] FLAG: --registry-qps="5"
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933819    6011 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 00:39:17.940978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933828    6011 flags.go:64] FLAG: --reserved-memory=""
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933832    6011 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933835    6011 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933855    6011 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933860    6011 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933863    6011 flags.go:64] FLAG: --runonce="false"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933866    6011 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933869    6011 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933873    6011 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933876    6011 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933879    6011 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933882    6011 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933885    6011 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933888    6011 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933891    6011 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933893    6011 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933896    6011 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933899    6011 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933902    6011 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933905    6011 flags.go:64] FLAG: --system-cgroups=""
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933908    6011 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933913    6011 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933918    6011 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933921    6011 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933932    6011 flags.go:64] FLAG: --tls-min-version=""
Oct 30 00:39:17.942043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933935    6011 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933938    6011 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933941    6011 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933944    6011 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933947    6011 flags.go:64] FLAG: --v="2"
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933951    6011 flags.go:64] FLAG: --version="false"
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933957    6011 flags.go:64] FLAG: --vmodule=""
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933961    6011 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.933970    6011 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934158    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934163    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934166    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934182    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934186    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934189    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934191    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934194    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934197    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934199    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934202    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934204    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.942814 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934207    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934210    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934214    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934218    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934220    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934223    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934226    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934228    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934232    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934235    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934237    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934240    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934243    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934245    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934248    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934251    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934253    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934258    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934260    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934263    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.943358 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934272    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934275    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934278    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934281    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934283    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934286    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934288    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934290    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934293    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934295    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934298    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934300    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934303    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934306    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934308    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934311    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934313    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934316    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934318    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.943841 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934321    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934324    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934327    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934329    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934332    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934335    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934337    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934340    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934342    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934345    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934349    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934351    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934353    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934356    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934364    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934367    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934370    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934372    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934374    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934377    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.944477 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934380    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934382    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934386    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934389    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934393    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934396    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934399    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934401    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934404    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934406    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934409    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934411    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934414    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934420    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934422    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934425    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934427    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934430    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934432    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.945112 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934434    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934437    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934440    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934443    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934446    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934448    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934451    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934453    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934461    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934464    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934467    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934469    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934472    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934475    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934477    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934480    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.945734 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.934505    6011 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934555    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934560    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934563    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934565    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934568    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934572    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934576    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934579    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934581    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934584    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934588    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934591    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934593    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934596    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934598    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934601    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934604    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934606    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934610    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.946119 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934613    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934616    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934618    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934620    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934623    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934631    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934634    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934637    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934640    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934642    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934645    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934647    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934650    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934652    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934655    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934657    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934660    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934663    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934665    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.946670 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934668    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934670    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934673    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934675    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934679    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934681    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934684    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934686    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934689    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934691    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934694    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934697    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934700    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934709    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934712    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934715    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934717    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934720    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934722    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.947205 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934732    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934735    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934737    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934740    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934742    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934745    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934747    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934751    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934754    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934757    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934759    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934762    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934764    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934767    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934780    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934784    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934788    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934793    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934796    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.947800 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934798    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934801    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934804    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934806    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934814    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934817    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934822    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934825    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934827    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934830    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934832    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934835    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934837    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934840    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934848    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934851    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934854    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934857    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934860    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934862    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.948524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934865    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934867    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934870    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934872    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934875    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934877    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934879    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934882    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934884    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.934887    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.939097    6011 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.939111    6011 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.939125    6011 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.939130    6011 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939213    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939218    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939222    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939224    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939227    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.949233 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939230    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939233    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939235    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939238    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939241    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939243    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939246    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939249    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939252    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939255    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939257    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939260    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939262    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939265    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939267    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939270    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939273    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939275    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939278    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.949686 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939281    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939284    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939286    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939289    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939292    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939294    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939297    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939300    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939303    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939307    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939311    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939314    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939318    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939322    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939324    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939327    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939329    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939332    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939334    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.950245 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939337    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939340    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939342    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939345    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939347    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939350    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939353    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939355    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939358    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939360    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939363    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939365    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939368    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939370    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939373    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939376    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939379    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939382    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939384    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939387    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.950860 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939390    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939392    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939395    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939397    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939400    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939402    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939405    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939407    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939410    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939412    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939415    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939418    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939420    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939423    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939426    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939429    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939431    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939434    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939436    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.951576 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939439    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939441    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939444    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939447    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939449    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939452    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939454    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939457    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939459    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939462    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939465    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939467    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939470    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939472    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939475    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939477    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939480    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939482    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939485    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939487    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.952095 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939490    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939492    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939494    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939497    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.939524    6011 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939662    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939667    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939671    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939674    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939678    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939681    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939683    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939685    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939688    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939690    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.952738 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939693    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939695    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939698    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939700    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939703    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939705    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939708    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939711    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939713    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939715    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939718    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939720    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939723    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939725    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939728    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939730    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939733    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939735    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939738    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939740    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.953304 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939743    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939745    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939748    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939751    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939753    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939756    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939758    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939761    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939764    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939766    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939769    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939771    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939773    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939776    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939778    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939781    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939783    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939786    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939788    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.954025 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939791    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939794    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939797    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939799    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939802    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939804    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939807    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939809    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939811    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939814    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939816    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939819    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939821    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939824    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939826    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939830    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939834    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939837    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939839    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.954657 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939842    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939845    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939847    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939850    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939853    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939855    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939857    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939859    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939862    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939864    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939867    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939869    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939872    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939875    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939877    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939880    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939882    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939884    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939895    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939898    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.955338 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939901    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939904    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939906    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939909    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939911    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939914    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939916    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939918    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939921    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939924    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939926    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939929    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939931    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939934    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939936    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939939    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939942    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.955896 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.939944    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.939968    6011 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940002    6011 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940006    6011 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940010    6011 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940015    6011 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940018    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940021    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940023    6011 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940026    6011 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940028    6011 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940031    6011 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940033    6011 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940036    6011 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940038    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940041    6011 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 00:39:17.956312 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940043    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940046    6011 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940048    6011 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940051    6011 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940053    6011 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940056    6011 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940058    6011 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940061    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940063    6011 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940066    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940068    6011 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940070    6011 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940073    6011 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940076    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940078    6011 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940081    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940083    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940086    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940088    6011 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940091    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 00:39:17.956704 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940093    6011 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940096    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940098    6011 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940101    6011 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940104    6011 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940111    6011 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940114    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940117    6011 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940119    6011 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940122    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940124    6011 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940127    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940129    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940132    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940134    6011 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940137    6011 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940139    6011 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940142    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940144    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 00:39:17.957306 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940147    6011 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940149    6011 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940152    6011 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940154    6011 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940156    6011 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940159    6011 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940161    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940164    6011 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940166    6011 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940169    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940185    6011 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940187    6011 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940190    6011 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940192    6011 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940195    6011 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940197    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940200    6011 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940202    6011 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940206    6011 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 00:39:17.957772 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940209    6011 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940212    6011 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940214    6011 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940217    6011 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940219    6011 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940221    6011 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940224    6011 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940227    6011 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940229    6011 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940232    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940234    6011 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940237    6011 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940239    6011 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940241    6011 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940244    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940246    6011 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940249    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940251    6011 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940254    6011 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 00:39:17.959914 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940256    6011 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940259    6011 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940261    6011 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940263    6011 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940266    6011 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940269    6011 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940272    6011 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940274    6011 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940277    6011 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940281    6011 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940284    6011 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940287    6011 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940290    6011 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940293    6011 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:39:17.940297    6011 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.940345    6011 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.941809    6011 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.941882    6011 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.942369    6011 server.go:1014] "Starting client certificate rotation"
Oct 30 00:39:17.960547 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.942498    6011 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.942547    6011 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 00:33:06 +0000 UTC" deadline="2028-06-04 21:33:23.783549889 +0000 UTC"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.942573    6011 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14012h54m5.840978547s"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.942806    6011 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.943882    6011 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.945110    6011 log.go:25] "Validated CRI v1 runtime API"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.947103    6011 log.go:25] "Validated CRI v1 image API"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.947921    6011 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 00:39:17.961098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.950805    6011 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 ab451a2b-7a30-4408-bec0-cb085eaaf3c4:/dev/nvme0n1p3 cc0a2405-f468-4b26-9bea-30b0b7254fce:/dev/nvme0n1p4]
Oct 30 00:39:17.961385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.950826    6011 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf/userdata/shm major:0 minor:258 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/250ca245080b48e3808e3d34afcb8a16f77cc0ba2da261f1d9a89a58d9493dd1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/250ca245080b48e3808e3d34afcb8a16f77cc0ba2da261f1d9a89a58d9493dd1/userdata/shm major:0 minor:303 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/26383512cc36d898f8de6229f9f4f6830fcac06d7fba3ffa229293be41790096/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/26383512cc36d898f8de6229f9f4f6830fcac06d7fba3ffa229293be41790096/userdata/shm major:0 minor:87 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2970c24f126574f409849126cf31e82193b6f5d93e03621186de59951da5ae7b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2970c24f126574f409849126cf31e82193b6f5d93e03621186de59951da5ae7b/userdata/shm major:0 minor:307 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/332f610b2772c7b3713325f1b5c74e11d9b9ae303f17c529b59144f5e12b55ed/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/332f610b2772c7b3713325f1b5c74e11d9b9ae303f17c529b59144f5e12b55ed/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/36918762d7bdd40bd5bcdc2e4d93c8a059f2e794161144f98f7f1b44c59213f5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/36918762d7bdd40bd5bcdc2e4d93c8a059f2e794161144f98f7f1b44c59213f5/userdata/shm major:0 minor:498 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3cc4adb2181f5cb23a4bba9093aaa1d5108e18d8038a46f406bb53f9ecbbcb29/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3cc4adb2181f5cb23a4bba9093aaa1d5108e18d8038a46f406bb53f9ecbbcb29/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/412046e464fe9bfdfcf1446b319fce0f33520fece5e515c677d3dac5e53c044d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/412046e464fe9bfdfcf1446b319fce0f33520fece5e515c677d3dac5e53c044d/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4708b81382efc61bab8901a2614a38bdef1da5c138e4f6a804171929ff55c57e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4708b81382efc61bab8901a2614a38bdef1da5c138e4f6a804171929ff55c57e/userdata/shm major:0 minor:271 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5fd88bbcea9ab74151f52abd18f825194212aa80873e44b6c7d9971d3a86dad5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5fd88bbcea9ab74151f52abd18f825194212aa80873e44b6c7d9971d3a86dad5/userdata/shm major:0 minor:505 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e/userdata/shm major:0 minor:383 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8bc0f39b14cac44d07cb4c31e4875b76787072021f455fdeed3132d74efb6382/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8bc0f39b14cac44d07cb4c31e4875b76787072021f455fdeed3132d74efb6382/userdata/shm major:0 minor:374 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9388184f99755682042b0e2b0aa92546c6e7f41c3ebc22d89e6ef0c6f3789ff3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9388184f99755682042b0e2b0aa92546c6e7f41c3ebc22d89e6ef0c6f3789ff3/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/956ecf285f0d4ec7a0a2695c9b7e754359909d3deee4b6bf9f3d12408428c4b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/956ecf285f0d4ec7a0a2695c9b7e754359909d3deee4b6bf9f3d12408428c4b8/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a3fc8b43d3536f447460a589b34a6ba1cbe1c4da07574f8e4234d1fddb733ab9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a3fc8b43d3536f447460a589b34a6ba1cbe1c4da07574f8e4234d1fddb733ab9/userdata/shm major:0 minor:251 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aa5b97fc6fd53b4d0b34492332c304107b3aa0a60224b259b757ab9c3ef6e286/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aa5b97fc6fd53b4d0b34492332c304107b3aa0a60224b259b757ab9c3ef6e286/userdata/shm major:0 minor:301 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aa7481591b6759b61b735054248a39b3f74d0b04eed2963b1f8fdfb4b13c9e5d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aa7481591b6759b61b735054248a39b3f74d0b04eed2963b1f8fdfb4b13c9e5d/userdata/shm major:0 minor:496 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aca378dc272d1049e9e21eaadf0d6d9cc658489b072275ba9f35c262000beae3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aca378dc272d1049e9e21eaadf0d6d9cc658489b072275ba9f35c262000beae3/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ba4a7524aa24ad2414db1fc9a78d51a2c0ef6ea199b826bebfa9a28c0d92e11d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ba4a7524aa24ad2414db1fc9a78d51a2c0ef6ea199b826bebfa9a28c0d92e11d/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d501239df6e20428a1af1daef082cb9622bcff5d5d981d348a66970a6e53509f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d501239df6e20428a1af1daef082cb9622bcff5d5d981d348a66970a6e53509f/userdata/shm major:0 minor:74 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d94b07be11a7323427254464954276c4ad0efdb139aef187ea01b04a44e71af0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d94b07be11a7323427254464954276c4ad0efdb139aef187ea01b04a44e71af0/userdata/shm major:0 minor:206 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/db67bdbb04cc28a261d52554b33192b8da20975fe2ef2cf082ea275dcaca59a0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/db67bdbb04cc28a261d52554b33192b8da20975fe2ef2cf082ea275dcaca59a0/userdata/shm major:0 minor:530 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e0e0d4c950ce469ddd0e02a2c7a0f064ad98d31f34da87edb90b76ad7cdd9865/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e0e0d4c950ce469ddd0e02a2c7a0f064ad98d31f34da87edb90b76ad7cdd9865/userdata/shm major:0 minor:409 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3/userdata/shm major:0 minor:48 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/eede4fd62fdeecb71c517704956e1c4ad7154c9df215916ed87b10bc3f78b642/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/eede4fd62fdeecb71c517704956e1c4ad7154c9df215916ed87b10bc3f78b642/userdata/shm major:0 minor:448 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117/userdata/shm major:0 minor:340 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fa0428cff534eb96c9e1ccb45c7c8f2fcceccb04f1d581b08392a6aedd113040/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fa0428cff534eb96c9e1ccb45c7c8f2fcceccb04f1d581b08392a6aedd113040/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volumes/kubernetes.io~projected/kube-api-access-8zfkx:{mountpoint:/var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volumes/kubernetes.io~projected/kube-api-access-8zfkx major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~projected/kube-api-access-ct2s6:{mountpoint:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~projected/kube-api-access-ct2s6 major:0 minor:406 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:404 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:402 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:403 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~projected/kube-api-access-8229s:{mountpoint:/var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~projected/kube-api-access-8229s major:0 minor:531 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:528 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:529 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d14ce64-f501-4c56-aeff-137f931bfea9/volumes/kubernetes.io~projected/kube-api-access-h7dcp:{mountpoint:/var/lib/kubelet/pods/0d14ce64-f501-4c56-aeff-137f931bfea9/volumes/kubernetes.io~projected/kube-api-access-h7dcp major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0d14ce64-f501-4c56-aeff-137f931bfea9/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/0d14ce64-f501-4c56-aeff-137f931bfea9/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~projected/kube-api-access-6qms8:{mountpoint:/var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~projected/kube-api-access-6qms8 major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:296 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/alertmanager/9 major:0 minor:332 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/config-reloader/6 major:0 minor:332 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:332 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~empty-dir/config-out major:0 minor:331 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~projected/kube-api-access-fn4jb:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~projected/kube-api-access-fn4jb major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~projected/tls-assets major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:329 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/config-volume major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:326 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:319 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:328 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/web-config major:0 minor:332 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~projected/kube-api-access-clvh4:{mountpoint:/var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~projected/kube-api-access-clvh4 major:0 minor:270 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:269 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24b870b7-104e-44b7-b15a-8db848e34bfc/volumes/kubernetes.io~projected/kube-api-access-bngtx:{mountpoint:/var/lib/kubelet/pods/24b870b7-104e-44b7-b15a-8db848e34bfc/volumes/kubernetes.io~projected/kube-api-access-bngtx major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~projected/kube-api-access-lmtz7:{mountpoint:/var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~projected/kube-api-access-lmtz7 major:0 minor:382 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:381 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:380 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a1157b8-e332-4163-b20a-1edbbe85f786/volumes/kubernetes.io~projected/kube-api-access-7xzw8:{mountpoint:/var/lib/kubelet/pods/3a1157b8-e332-4163-b20a-1edbbe85f786/volumes/kubernetes.io~projected/kube-api-access-7xzw8 major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3a1157b8-e332-4163-b20a-1edbbe85f786/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/3a1157b8-e332-4163-b20a-1edbbe85f786/volumes/kubernetes.io~secret/metrics-certs major:0 minor:495 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~projected/kube-api-access-95rd7:{mountpoint:/var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~projected/kube-api-access-95rd7 major:0 minor:294 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:293 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:289 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~projected/kube-api-access-4zhgq:{mountpoint:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~projected/kube-api-access-4zhgq major:0 minor:367 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:364 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/57d0d345-d190-4a7b-b61c-f2f2da64ffc8/volumes/kubernetes.io~projected/kube-api-access-zflwb:{mountpoint:/var/lib/kubelet/pods/57d0d345-d190-4a7b-b61c-f2f2da64ffc8/volumes/kubernetes.io~projected/kube-api-access-zflwb major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5a3f42a4-9d45-4d56-952c-4dd04bb47261/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/5a3f42a4-9d45-4d56-952c-4dd04bb47261/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/5e25b748-6ffe-488b-a210-24f1b4d1ce57/volumes/kubernetes.io~secret/tls-certificates:{mountpoint:/var/lib/kubelet/pods/5e25b748-6ffe-488b-a210-24f1b4d1ce57/volumes/kubernetes.io~secret/tls-certificates major:0 minor:246 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~projected/kube-api-access-qwlfj:{mountpoint:/var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~projected/kube-api-access-qwlfj major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:255 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/66b60042-598e-49bb-8de0-009bd4c1bd6b/volumes/kubernetes.io~projected/kube-api-access-ts7xc:{mountpoint:/var/lib/kubelet/pods/66b60042-598e-49bb-8de0-009bd4c1bd6b/volumes/kubernetes.io~projected/kube-api-access-ts7xc major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~empty-dir/tmp major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~projected/kube-api-access-h7qlk:{mountpoint:/var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~projected/kube-api-access-h7qlk major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ebd1a13-21bc-4de8-aaad-4e289981ee10/volumes/kubernetes.io~projected/kube-api-access-ndnng:{mountpoint:/var/lib/kubelet/pods/6ebd1a13-21bc-4de8-aaad-4e289981ee10/volumes/kubernetes.io~projected/kube-api-access-ndnng major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6ebd1a13-21bc-4de8-aaad-4e289981ee10/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/6ebd1a13-21bc-4de8-aaad-4e289981ee10/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3/volumes/kubernetes.io~projected/kube-api-access-nsrkq:{mountpoint:/var/lib/kubelet/pods/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3/volumes/kubernetes.io~projected/kube-api-access-nsrkq major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2a1b468-478c-445d-af74-05dac392aec7/volumes/kubernetes.io~projected/kube-api-access-dmcw7:{mountpoint:/var/lib/kubelet/pods/a2a1b468-478c-445d-af74-05dac392aec7/volumes/kubernetes.io~projected/kube-api-access-dmcw7 major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~projected/kube-api-access-5497b:{mountpoint:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~projected/kube-api-access-5497b major:0 minor:440 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/federate-client-tls:{mountpoint:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/federate-client-tls major:0 minor:447 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/secret-telemeter-client:{mountpoint:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/secret-telemeter-client major:0 minor:445 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config major:0 minor:446 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/telemeter-client-tls:{mountpoint:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/telemeter-client-tls major:0 minor:444 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b5d71931-979a-4170-abe4-d22055d75da2/volumes/kubernetes.io~projected/kube-api-access-tvxcg:{mountpoint:/var/lib/kubelet/pods/b5d71931-979a-4170-abe4-d22055d75da2/volumes/kubernetes.io~projected/kube-api-access-tvxcg major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:491 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd83da39-c295-436b-a07c-4c41c68de6be/volumes/kubernetes.io~projected/kube-api-access-bc4fj:{mountpoint:/var/lib/kubelet/pods/cd83da39-c295-436b-a07c-4c41c68de6be/volumes/kubernetes.io~projected/kube-api-access-bc4fj major:0 minor:500 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~projected/kube-api-access-5h74d:{mountpoint:/var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~projected/kube-api-access-5h74d major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:297 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:298 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb95c197-005f-4deb-b563-107e7e0aff33/volumes/kubernetes.io~projected/kube-api-access-jpqk8:{mountpoint:/var/lib/kubelet/pods/fb95c197-005f-4deb-b563-107e7e0aff33/volumes/kubernetes.io~projected/kube-api-access-jpqk8 major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fb95c197-005f-4deb-b563-107e7e0aff33/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/fb95c197-005f-4deb-b563-107e7e0aff33/volumes/kubernetes.io~secret/metrics-tls major:0 minor:197 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/99e765ff8d8ebe08e4aac391394415257d692fbb95a67232c07889c51e9f6a8a/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/e94d42575d98e180fef7ab3fe04e34c1559ff89fd6869d28590b19867b05674a/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/ccdbe23c4db5387f7114e027422913c7eba6591c7820a75ac7cfa73974af8785/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/b06577ee62a4033771695121d3ba3e2a6b202f723e07254f0693f0521bb03042/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-110:{mountpoint:/var/lib/containers/storage/overlay/52afba0c85340ab82f9f1d55717450b405c4da5db1a3554ba4792444e19d5211/merged major:0 minor:110 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/034c23cf320f72ba557041e058348d7c1ea4a2a10651155a57f05ce5c15b5b80/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/c259803412cceba69516d7161b5760680c70467afe7498a8e9b84406d0797637/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/79e90fef4d68c69de3eb1d0c37cf5ede57db9f5f9fb98e174024ded12470963d/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-129:{mountpoint:/var/lib/containers/storage/overlay/6fe368ccd86ef5b2676450449b3f5aecafe9c391d765246eb44beabfc4eda19b/merged major:0 minor:129 fsType:overlay blockSize:0} overlay_0-137:{mountpoint:/var/lib/containers/storage/overlay/97a6c12454a3b0f296c92e7db0d10ef75ada8c8ab367ffb80593c10259fa0d26/merged major:0 minor:137 fsType:overlay blockSize:0} overlay_0-142:{mountpoint:/var/lib/containers/storage/overlay/40f3ca3af7bdc0c39dc11c14ac7ec0f341b1d5e82147ecf151ddeeb013fc26fb/merged major:0 minor:142 fsType:overlay blockSize:0} overlay_0-143:{mountpoint:/var/lib/containers/storage/overlay/dfac43ac7322b0b96a7899de109ee5b4ae28578f3c132a404fa33a19e535aefd/merged major:0 minor:143 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/c070ab3d9d61c1462336ef78e75cef3be9c5cf5368ecf0bcae6a13b3bd17a47b/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/e01a63c7e068865ca09926ecba958b15e0d6671f54bf62a2853e0e8f31734a7c/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/d72ec678fe2a0979ac25c0da151bfa603808bfb3330a641552947e58ab9943dd/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/da96165beb177f280e2a2c5b03081f30a3428aed773e487cb72c9d32ac644098/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/c9f2c3c020662863ca136a78602ac95487540d1122a3ace035a63571ce9479e9/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/0483fa14cd4fb7f28127212aa640f84c400c7c1ef09e01b3338f46fe46fcc4e4/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/eb9e941868ee747f7e3754d8721c361b962c6def65139b8f5d6d9882f7bae6fe/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/5a517a8017205a9a40c99de48330910dc51e1b4724b8d6ad49f15e2a8dda99dd/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/3414f7e88165be6603f04dc15dccb1a84cd4f3cf3b604f934a696853f1c200da/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/2b4b7b8cf3fae804b0a76bef768e3fd6f13608207326eca0564f4442f0ffc87d/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/02f4a5fd9eed9d952def2f8bfda63529ea4ebe42628005ace1ea27924d6dea3a/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/5c06b5a6b9a505338552af747417e4ba75dfd8e000b565e02cb434896f5aa072/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/61485210c67039fd264c449dd85532babdd4337755606edb5f0f4ce982372498/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-219:{mountpoint:/var/lib/containers/storage/overlay/83b7db2eff81ac277ba54706eed87614d625e8706fe2c104a3c460b0162b2bb4/merged major:0 minor:219 fsType:overlay blockSize:0} overlay_0-225:{mountpoint:/var/lib/containers/storage/overlay/6ef235ca62b94c4d432b1adff519fe9f6c28c56d0fe39cd2bcefbbbf9f89d90c/merged major:0 minor:225 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/659bccfdac662d153fa32ed4c2f7d1bf32b7ec23333a201ff5f63cacec6e5839/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/eaa19482262e618ae8bb9dfa559c3a5ca276820668b12d152d90c5360abdba20/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/bf2a81296ecdaa4ded3b5b7caf715525a5ac8189bcc7d89acb4809f29ca888a4/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/b04946612a3c567a355808b7258ed97bca3849694eca1ad7a55844596f985705/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/69a75964f70c916ccc2943d1949b0bd2b743440588ed617d607af212a611d1e0/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/346895516e273823e4fd5d704e7a40fcc837d25491726191270f6de2b5c6f9df/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-273:{mountpoint:/var/lib/containers/storage/overlay/26422bba18ddd4b0e426bdecb29511b47226be30e1e086838579471a37c7c362/merged major:0 minor:273 fsType:overlay blockSize:0} overlay_0-275:{mountpoint:/var/lib/containers/storage/overlay/6a7fec23a27c8fe974050c745fe79940164a6daea4de9f3eb8d5569f6a6dc37e/merged major:0 minor:275 fsType:overlay blockSize:0} overlay_0-277:{mountpoint:/var/lib/containers/storage/overlay/c233de249430115377a35877bacfc2971cad524a619c6a0fce80be207947347b/merged major:0 minor:277 fsType:overlay blockSize:0} overlay_0-283:{mountpoint:/var/lib/containers/storage/overlay/38bc9301c8e9e03666f4a82048dc5c684f938159466086aa75bcf589d3eac333/merged major:0 minor:283 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/3eb3e2fec98c1926004bca01dd98c925c69d53338d070ce779f51f28b3a1d9a7/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/fedf2878f48dc8bdaf7b2f41d7a126f317be7bdc38074e23b439877e12e9ea5e/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/5b3bf2bcd5d9ab931a31eca3c59427e491182c7d5d0c5eb68248194733b7ff88/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-313:{mountpoint:/var/lib/containers/storage/overlay/ae1a102d16fb21ac4985789117b4fb04af85c61851b5b25714286b25d1b5fa81/merged major:0 minor:313 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/aa9a3c922604d502cb6da9a50bf359886766092ea87bbff43b8349cd968287a7/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-334:{mountpoint:/var/lib/containers/storage/overlay/228ab3de1ac28e9c66f252990e08153d3c009d16b2e5969f15ff349a8982327c/merged major:0 minor:334 fsType:overlay blockSize:0} overlay_0-336:{mountpoint:/var/lib/containers/storage/overlay/2f4d82969724aac7b24ce51334c9de155f7f3a4a7bad35641e0be92e37847cbf/merged major:0 minor:336 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/0cc0c3ff2ed409665975ee9497274a873fd5b572b20ad147609b5bdcf766da89/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-349:{mountpoint:/var/lib/containers/storage/overlay/8516bf0980d7aac45979ddaf85dd3a8d37bf6e27083a6609ff949647f0845f31/merged major:0 minor:349 fsType:overlay blockSize:0} overlay_0-351:{mountpoint:/var/lib/containers/storage/overlay/e7db4d06dc58a70aae5454fe35e983001118cc9e11d9ef734a0ffa3f952e0949/merged major:0 minor:351 fsType:overlay blockSize:0} overlay_0-365:{mountpoint:/var/lib/containers/storage/overlay/0eb77cd80492e0e8afb9cced5ab782593bfd397f69b37b16d8e6a3b653455eff/merged major:0 minor:365 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/36a6f6f8a67f2c00c0877211844a2626354f6f159f455f0f1fe1e3c337ef85d0/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-373:{mountpoint:/var/lib/containers/storage/overlay/1a021b03136e9e7eb89969e8ea6c618e9c6a35cf58301df3bdad18cb952d35df/merged major:0 minor:373 fsType:overlay blockSize:0} overlay_0-385:{mountpoint:/var/lib/containers/storage/overlay/41dcd65650747b0c76e5e1e056bdadb02c12339e8e5f1082d6318104d2adbdf3/merged major:0 minor:385 fsType:overlay blockSize:0} overlay_0-387:{mountpoint:/var/lib/containers/storage/overlay/6be23e5563b684fb571538e3fc83959cda95da963cb8f0368770514b91b171f0/merged major:0 minor:387 fsType:overlay blockSize:0} overlay_0-393:{mountpoint:/var/lib/containers/storage/overlay/c5a99d6d05190fc77d701f8796d54d73699ceb109f83780cb84c43ea914b9e05/merged major:0 minor:393 fsType:overlay blockSize:0} overlay_0-398:{mountpoint:/var/lib/containers/storage/overlay/f6fceb6367c14a0dad3a315812efe6277c9de3eeda23510a4dd6e0b8d353d682/merged major:0 minor:398 fsType:overlay blockSize:0} overlay_0-407:{mountpoint:/var/lib/containers/storage/overlay/be51706cf0cd50c3bd6045d5556cbb6fb1235e8e130bc4548444613f5f5b31f8/merged major:0 minor:407 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/5a3855c2e8d1009c74db166424dd843ee7cc879d28471e50ccc229fe25a7ebfd/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-417:{mountpoint:/var/lib/containers/storage/overlay/e0d149533a4601f1f44fcdf6792dfab077d650ac890da7edc3107c25adb10a6a/merged major:0 minor:417 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/8294e86636014695f1ea74c3530b9e33d25f1833dc016950f75a2f154be2c088/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/3fbba5cbd3abd3bf1a32f26ad40cdb3cd50b2be0edab68ad7c9d6f43c827e616/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-430:{mountpoint:/var/lib/containers/storage/overlay/f46448d762c7dd97951d37f94d4e05fb754012150e057c30103962018f1245a0/merged major:0 minor:430 fsType:overlay blockSize:0} overlay_0-435:{mountpoint:/var/lib/containers/storage/overlay/996b932d253264cfbf49ae2d2c559d7cb007140195c7fe747f8e2da1ac3c7f27/merged major:0 minor:435 fsType:overlay blockSize:0} overlay_0-450:{mountpoint:/var/lib/containers/storage/overlay/06d5f404b21fafaf2a642c4b7ad81cfcfd365375bdbf74142f2dab856e1d4c1e/merged major:0 minor:450 fsType:overlay blockSize:0} overlay_0-452:{mountpoint:/var/lib/containers/storage/overlay/de90284294bdb27acbe54cc82eaddf8d3249419211feca61cdcc950abd3cf983/merged major:0 minor:452 fsType:overlay blockSize:0} overlay_0-457:{mountpoint:/var/lib/containers/storage/overlay/e770791f6b79fc37ef9fadb510dcd776cdf4b8496f7f5decc7c524a002434583/merged major:0 minor:457 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/307b7d08c3feebff5235dbd1f866a03916c456ccf0e690414b0f806ed19258bf/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-467:{mountpoint:/var/lib/containers/storage/overlay/8294b3458136b722b582e7e2954d43b78956830461b432fe86a51087e952ce33/merged major:0 minor:467 fsType:overlay blockSize:0} overlay_0-469:{mountpoint:/var/lib/containers/storage/overlay/42ab745da2edaa37baf4c532e27b30a78fc97f16ffa5e5c7d3d350aa146d193a/merged major:0 minor:469 fsType:overlay blockSize:0} overlay_0-474:{mountpoint:/var/lib/containers/storage/overlay/3432238a1285addb059d9158fb6396c890be5c9f14de3dcb624f727734d92dcf/merged major:0 minor:474 fsType:overlay blockSize:0} overlay_0-480:{mountpoint:/var/lib/containers/storage/overlay/ea2ae59f30d5664e811fb572b46a39ceb040830db883a75e7aeced95a2f9c45e/merged major:0 minor:480 fsType:overlay blockSize:0} overlay_0-486:{mountpoint:/var/lib/containers/storage/overlay/d68a7c305ac0f9974fef89d6610c1fe12b79fa4d806c5150dc5f6ca9942d3208/merged major:0 minor:486 fsType:overlay blockSize:0} overlay_0-501:{mountpoint:/var/lib/containers/storage/overlay/1557c68e10f4d017752a04f030f1a50bab22f442843658f8385c8425da2dde94/merged major:0 minor:501 fsType:overlay blockSize:0} overlay_0-503:{mountpoint:/var/lib/containers/storage/overlay/20bb23e4831409369c1dee48f3c872c61321e353ac1f9c85b0f056a01aede551/merged major:0 minor:503 fsType:overlay blockSize:0} overlay_0-507:{mountpoint:/var/lib/containers/storage/overlay/29c3ae11dcefac9a22059d9b38c4b2af83e130b745cea9ce13a27437d16e6e7c/merged major:0 minor:507 fsType:overlay blockSize:0} overlay_0-509:{mountpoint:/var/lib/containers/storage/overlay/8fd5bbd7d6e0026e3be5f7fb5024e7d227898f4342ce670e3b8326a70268979e/merged major:0 minor:509 fsType:overlay blockSize:0} overlay_0-51:{mountpoint:/var/lib/containers/storage/overlay/61a1cfcb362573f06dcb8bb4d560a243923273675e0d38a680a52f24d36ca0a9/merged major:0 minor:51 fsType:overlay blockSize:0} overlay_0-511:{mountpoint:/var/lib/containers/storage/overlay/c5754333ecee8daf1e7238fa4772c422b709dfd7b974847676cf3e944ab82f40/merged major:0 minor:511 fsType:overlay blockSize:0} overlay_0-517:{mountpoint:/var/lib/containers/storage/overlay/d9712afffb7d2b7ff292b9a8845784929cd9feb9881ce9d1e9efd8c9d29fee80/merged major:0 minor:517 fsType:overlay blockSize:0} overlay_0-522:{mountpoint:/var/lib/containers/storage/overlay/2dd21eec143c2ac7965f63fd8eae37c759be7135d96cc5cc03ea14396b341b39/merged major:0 minor:522 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/fae7a3d3540b87a3e71866cd0af55f59828e0fb931dd5f573c03f0d3aa40bc87/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/5157870c5f3d8b0ac06a30380d172846de5de1f73252d74e87a2cb23e44032ed/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/160567d70a0b484cdc151243379164b5e20a550ddd1a1d7b9c8e129db422c9fe/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/1d1219bed42052e2428e65d951d82b9868bc08ea50ee76212ca3c604cb9d8c41/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/faf0aa172dadf9fba6cdb17fc9b607b9bde61ac0c9ff2d7bb8a40d8274c652f4/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/516b992b8bffffd94539730b8992ddf80caf1bf156df6e653077f3b324fcdd31/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/5e4f1ba10b22976d00a72cf4711f9c349a46063566ef9522e752b4a5f73a33ce/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/bc4f9dc0f376094f59731ec5d2372672dbe39e0148154e622748aa5a0a1d0bbe/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/7ea6dcb0067a07237a72ef2bc1b9c44558ba9014afd8a1455275ab9bd241b288/merged major:0 minor:96 fsType:overlay blockSize:0}]
Oct 30 00:39:17.961750 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.961284    6011 manager.go:217] Machine: {Timestamp:2026-10-30 00:39:17.960768407 +0000 UTC m=+0.063522369 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091093 MemoryCapacity:32812015616 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec298aa3bcb77a693f417c968382affd SystemUUID:ec298aa3-bcb7-7a69-3f41-7c968382affd BootID:7b8c16a6-3c54-45c8-98f5-71ef7377dd82 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3cc4adb2181f5cb23a4bba9093aaa1d5108e18d8038a46f406bb53f9ecbbcb29/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-110 DeviceMajor:0 DeviceMinor:110 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:491 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/26383512cc36d898f8de6229f9f4f6830fcac06d7fba3ffa229293be41790096/userdata/shm DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6ebd1a13-21bc-4de8-aaad-4e289981ee10/volumes/kubernetes.io~projected/kube-api-access-ndnng DeviceMajor:0 DeviceMinor:204 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:329 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-474 DeviceMajor:0 DeviceMinor:474 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/36918762d7bdd40bd5bcdc2e4d93c8a059f2e794161144f98f7f1b44c59213f5/userdata/shm DeviceMajor:0 DeviceMinor:498 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/57d0d345-d190-4a7b-b61c-f2f2da64ffc8/volumes/kubernetes.io~projected/kube-api-access-zflwb DeviceMajor:0 DeviceMinor:64 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:328 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:363 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-385 DeviceMajor:0 DeviceMinor:385 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-507 DeviceMajor:0 DeviceMinor:507 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:528 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/412046e464fe9bfdfcf1446b319fce0f33520fece5e515c677d3dac5e53c044d/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/66b60042-598e-49bb-8de0-009bd4c1bd6b/volumes/kubernetes.io~projected/kube-api-access-ts7xc DeviceMajor:0 DeviceMinor:63 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/5a3f42a4-9d45-4d56-952c-4dd04bb47261/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:56 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/332f610b2772c7b3713325f1b5c74e11d9b9ae303f17c529b59144f5e12b55ed/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:297 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-334 DeviceMajor:0 DeviceMinor:334 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:402 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:403 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:57 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:326 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3/userdata/shm DeviceMajor:0 DeviceMinor:48 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~projected/kube-api-access-clvh4 DeviceMajor:0 DeviceMinor:270 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:298 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e0e0d4c950ce469ddd0e02a2c7a0f064ad98d31f34da87edb90b76ad7cdd9865/userdata/shm DeviceMajor:0 DeviceMinor:409 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-435 DeviceMajor:0 DeviceMinor:435 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/24b870b7-104e-44b7-b15a-8db848e34bfc/volumes/kubernetes.io~projected/kube-api-access-bngtx DeviceMajor:0 DeviceMinor:66 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-277 DeviceMajor:0 DeviceMinor:277 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:362 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/secret-telemeter-client DeviceMajor:0 DeviceMinor:445 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-517 DeviceMajor:0 DeviceMinor:517 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~projected/kube-api-access-95rd7 DeviceMajor:0 DeviceMinor:294 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:360 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/federate-client-tls DeviceMajor:0 DeviceMinor:447 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-486 DeviceMajor:0 DeviceMinor:486 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-509 DeviceMajor:0 DeviceMinor:509 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-275 DeviceMajor:0 DeviceMinor:275 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-351 DeviceMajor:0 DeviceMinor:351 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:332 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/telemeter-client-tls DeviceMajor:0 DeviceMinor:444 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-480 DeviceMajor:0 DeviceMinor:480 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~projected/kube-api-access-8229s DeviceMajor:0 DeviceMinor:531 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:58 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a3fc8b43d3536f447460a589b34a6ba1cbe1c4da07574f8e4234d1fddb733ab9/userdata/shm DeviceMajor:0 DeviceMinor:251 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf/userdata/shm DeviceMajor:0 DeviceMinor:258 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:295 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:325 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:404 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/volumes/kubernetes.io~projected/kube-api-access-5h74d DeviceMajor:0 DeviceMinor:300 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8bc0f39b14cac44d07cb4c31e4875b76787072021f455fdeed3132d74efb6382/userdata/shm DeviceMajor:0 DeviceMinor:374 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e/userdata/shm DeviceMajor:0 DeviceMinor:383 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-398 DeviceMajor:0 DeviceMinor:398 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406007808 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/a2a1b468-478c-445d-af74-05dac392aec7/volumes/kubernetes.io~projected/kube-api-access-dmcw7 DeviceMajor:0 DeviceMinor:65 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/fb95c197-005f-4deb-b563-107e7e0aff33/volumes/kubernetes.io~projected/kube-api-access-jpqk8 DeviceMajor:0 DeviceMinor:203 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0d14ce64-f501-4c56-aeff-137f931bfea9/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:201 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:256 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:319 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:364 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-373 DeviceMajor:0 DeviceMinor:373 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3a1157b8-e332-4163-b20a-1edbbe85f786/volumes/kubernetes.io~projected/kube-api-access-7xzw8 DeviceMajor:0 DeviceMinor:60 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/68a4bb84-29e4-43b6-92df-7a4ea03ee820/volumes/kubernetes.io~projected/kube-api-access-h7qlk DeviceMajor:0 DeviceMinor:62 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9388184f99755682042b0e2b0aa92546c6e7f41c3ebc22d89e6ef0c6f3789ff3/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-137 DeviceMajor:0 DeviceMinor:137 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:332 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d94b07be11a7323427254464954276c4ad0efdb139aef187ea01b04a44e71af0/userdata/shm DeviceMajor:0 DeviceMinor:206 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:268 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:289 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~projected/kube-api-access-lmtz7 DeviceMajor:0 DeviceMinor:382 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-407 DeviceMajor:0 DeviceMinor:407 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-511 DeviceMajor:0 DeviceMinor:511 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fa0428cff534eb96c9e1ccb45c7c8f2fcceccb04f1d581b08392a6aedd113040/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/6ebd1a13-21bc-4de8-aaad-4e289981ee10/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:329 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/01d89188-b530-4373-9ea9-b58a4122c8cd/volumes/kubernetes.io~projected/kube-api-access-ct2s6 DeviceMajor:0 DeviceMinor:406 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-430 DeviceMajor:0 DeviceMinor:430 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~projected/kube-api-access-5497b DeviceMajor:0 DeviceMinor:440 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/3a1157b8-e332-4163-b20a-1edbbe85f786/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:495 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/cd83da39-c295-436b-a07c-4c41c68de6be/volumes/kubernetes.io~projected/kube-api-access-bc4fj DeviceMajor:0 DeviceMinor:500 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/5e25b748-6ffe-488b-a210-24f1b4d1ce57/volumes/kubernetes.io~secret/tls-certificates DeviceMajor:0 DeviceMinor:246 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:255 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:296 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~projected/kube-api-access-fn4jb DeviceMajor:0 DeviceMinor:333 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-349 DeviceMajor:0 DeviceMinor:349 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-365 DeviceMajor:0 DeviceMinor:365 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aca378dc272d1049e9e21eaadf0d6d9cc658489b072275ba9f35c262000beae3/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-225 DeviceMajor:0 DeviceMinor:225 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-51 DeviceMajor:0 DeviceMinor:51 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-393 DeviceMajor:0 DeviceMinor:393 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-457 DeviceMajor:0 DeviceMinor:457 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volumes/kubernetes.io~projected/kube-api-access-8zfkx DeviceMajor:0 DeviceMinor:59 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ba4a7524aa24ad2414db1fc9a78d51a2c0ef6ea199b826bebfa9a28c0d92e11d/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-219 DeviceMajor:0 DeviceMinor:219 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/162c7603-5367-4d79-8ff6-23b05743cf35/volumes/kubernetes.io~projected/kube-api-access-6qms8 DeviceMajor:0 DeviceMinor:299 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:331 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-336 DeviceMajor:0 DeviceMinor:336 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/eede4fd62fdeecb71c517704956e1c4ad7154c9df215916ed87b10bc3f78b642/userdata/shm DeviceMajor:0 DeviceMinor:448 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/b5d71931-979a-4170-abe4-d22055d75da2/volumes/kubernetes.io~projected/kube-api-access-tvxcg DeviceMajor:0 DeviceMinor:67 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0d14ce64-f501-4c56-aeff-137f931bfea9/volumes/kubernetes.io~projected/kube-api-access-h7dcp DeviceMajor:0 DeviceMinor:202 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-313 DeviceMajor:0 DeviceMinor:313 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~projected/kube-api-access-4zhgq DeviceMajor:0 DeviceMinor:367 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fb95c197-005f-4deb-b563-107e7e0aff33/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:197 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aa5b97fc6fd53b4d0b34492332c304107b3aa0a60224b259b757ab9c3ef6e286/userdata/shm DeviceMajor:0 DeviceMinor:301 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:380 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-387 DeviceMajor:0 DeviceMinor:387 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-450 DeviceMajor:0 DeviceMinor:450 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4708b81382efc61bab8901a2614a38bdef1da5c138e4f6a804171929ff55c57e/userdata/shm DeviceMajor:0 DeviceMinor:271 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:361 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-503 DeviceMajor:0 DeviceMinor:503 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-129 DeviceMajor:0 DeviceMinor:129 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1f7acedd-ae54-4ad5-bef2-da836c8ef78e/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:269 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-469 DeviceMajor:0 DeviceMinor:469 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-501 DeviceMajor:0 DeviceMinor:501 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-143 DeviceMajor:0 DeviceMinor:143 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4d53587f-0770-4a39-a369-dcc257c949f5/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:356 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-467 DeviceMajor:0 DeviceMinor:467 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3/volumes/kubernetes.io~projected/kube-api-access-nsrkq DeviceMajor:0 DeviceMinor:61 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-273 DeviceMajor:0 DeviceMinor:273 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/47f11449-4b9d-44df-8fed-68f31ded1601/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:293 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:381 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5fd88bbcea9ab74151f52abd18f825194212aa80873e44b6c7d9971d3a86dad5/userdata/shm DeviceMajor:0 DeviceMinor:505 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes/kubernetes.io~projected/kube-api-access-qwlfj DeviceMajor:0 DeviceMinor:257 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/ae0fed46-f827-4a14-879c-b0c4a4fa7db8/volumes/kubernetes.io~secret/secret-telemeter-client-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:446 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aa7481591b6759b61b735054248a39b3f74d0b04eed2963b1f8fdfb4b13c9e5d/userdata/shm DeviceMajor:0 DeviceMinor:496 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/01a7c97e-b248-4348-a07a-77652f9ce3c9/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-283 DeviceMajor:0 DeviceMinor:283 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:332 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117/userdata/shm DeviceMajor:0 DeviceMinor:340 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-522 DeviceMajor:0 DeviceMinor:522 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:327 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-417 DeviceMajor:0 DeviceMinor:417 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-452 DeviceMajor:0 DeviceMinor:452 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:529 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/db67bdbb04cc28a261d52554b33192b8da20975fe2ef2cf082ea275dcaca59a0/userdata/shm DeviceMajor:0 DeviceMinor:530 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d501239df6e20428a1af1daef082cb9622bcff5d5d981d348a66970a6e53509f/userdata/shm DeviceMajor:0 DeviceMinor:74 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/956ecf285f0d4ec7a0a2695c9b7e754359909d3deee4b6bf9f3d12408428c4b8/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-142 DeviceMajor:0 DeviceMinor:142 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/250ca245080b48e3808e3d34afcb8a16f77cc0ba2da261f1d9a89a58d9493dd1/userdata/shm DeviceMajor:0 DeviceMinor:303 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2970c24f126574f409849126cf31e82193b6f5d93e03621186de59951da5ae7b/userdata/shm DeviceMajor:0 DeviceMinor:307 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:330 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:332 Capacity:31633416192 Type:vfs Inodes:4005373 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:207928c3644e47e MacAddress:7e:04:89:06:48:0b Speed:10000 Mtu:8901} {Name:2970c24f126574f MacAddress:ae:6f:3d:a5:a0:8b Speed:10000 Mtu:8901} {Name:332f610b2772c7b MacAddress:56:7c:21:66:bc:fd Speed:10000 Mtu:8901} {Name:36918762d7bdd40 MacAddress:3a:7a:c9:19:ce:00 Speed:10000 Mtu:8901} {Name:4708b81382efc61 MacAddress:e6:fa:95:77:e1:eb Speed:10000 Mtu:8901} {Name:5fd88bbcea9ab74 MacAddress:6a:17:df:98:6a:da Speed:10000 Mtu:8901} {Name:6f7b32e0fb17a75 MacAddress:1e:64:7e:f6:2b:49 Speed:10000 Mtu:8901} {Name:8bc0f39b14cac44 MacAddress:32:32:0e:e8:4e:19 Speed:10000 Mtu:8901} {Name:a3fc8b43d3536f4 MacAddress:6a:c1:1a:dd:eb:6e Speed:10000 Mtu:8901} {Name:aa5b97fc6fd53b4 MacAddress:1a:38:10:f8:8d:ce Speed:10000 Mtu:8901} {Name:aa7481591b6759b MacAddress:be:ea:6c:cf:86:40 Speed:10000 Mtu:8901} {Name:aca378dc272d104 MacAddress:ca:a7:7d:39:0e:8d Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:f5:4f:5f:19 Speed:0 Mtu:9001} {Name:br-int MacAddress:fa:fe:ce:9f:d1:20 Speed:0 Mtu:8901} {Name:d94b07be11a7323 MacAddress:9e:98:51:1e:63:33 Speed:10000 Mtu:8901} {Name:db67bdbb04cc28a MacAddress:2a:3b:06:4c:de:79 Speed:10000 Mtu:8901} {Name:e0e0d4c950ce469 MacAddress:3e:51:ea:ff:1a:5e Speed:10000 Mtu:8901} {Name:eede4fd62fdeecb MacAddress:da:a7:30:61:99:bc Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:f5:4f:5f:19 Speed:0 Mtu:9001} {Name:f45aa391f5719f6 MacAddress:3e:61:c4:b8:78:78 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:22:34:f6:5f:b5:f5 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:82:b4:cc:d8:04:73 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812015616 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 00:39:17.961976 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.961755    6011 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 00:39:17.961976 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.961801    6011 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 00:39:17.962099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962077    6011 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 00:39:17.962254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962104    6011 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-254.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 00:39:17.962298 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962257    6011 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 00:39:17.962298 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962265    6011 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 00:39:17.962298 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962275    6011 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:39:17.962298 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962288    6011 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 00:39:17.962404 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962380    6011 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 00:39:17.962563 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962553    6011 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 00:39:17.962681 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962673    6011 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 00:39:17.962718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962711    6011 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 00:39:17.962747 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962726    6011 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 00:39:17.962747 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962737    6011 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 00:39:17.962747 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962745    6011 kubelet.go:405] "Adding apiserver pod source"
Oct 30 00:39:17.962828 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.962754    6011 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 00:39:17.963699 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.963686    6011 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 00:39:17.963699 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.963700    6011 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:39:17.964243 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.964155    6011 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 00:39:17.967813 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.967799    6011 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 00:39:17.967813 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.967813    6011 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 00:39:17.967917 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.967912    6011 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 00:39:17.968105 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968094    6011 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 00:39:17.968136 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968123    6011 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 00:39:17.968269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968256    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 00:39:17.968309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968280    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 00:39:17.968309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968290    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 00:39:17.968309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968296    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 00:39:17.968309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968302    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 00:39:17.968309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968308    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968314    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968319    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968327    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968332    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968340    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968349    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968375    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 00:39:17.968498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.968381    6011 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 00:39:17.972125 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.972108    6011 server.go:1263] "Started kubelet"
Oct 30 00:39:17.972222 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.972154    6011 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 00:39:17.972969 ip-10-0-131-254 systemd[1]: Started Kubernetes Kubelet.
Oct 30 00:39:17.973221 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.972236    6011 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 00:39:17.973221 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.973054    6011 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 00:39:17.973400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.973384    6011 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 00:39:17.973507 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.973493    6011 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.979297    6011 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.979321    6011 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.979393    6011 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 00:33:08 +0000 UTC" deadline="2028-06-23 19:16:47.502588759 +0000 UTC"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.979410    6011 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.979418    6011 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:17.979415    6011 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-131-254.ec2.internal\" not found"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.979551    6011 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 00:39:17.980307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.979414    6011 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14466h37m29.523177349s"
Oct 30 00:39:17.983038 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983004    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47f11449-4b9d-44df-8fed-68f31ded1601" volumeName="kubernetes.io/projected/47f11449-4b9d-44df-8fed-68f31ded1601-kube-api-access-95rd7" seLinuxMountContext=""
Oct 30 00:39:17.983038 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983037    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d14ce64-f501-4c56-aeff-137f931bfea9" volumeName="kubernetes.io/projected/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-api-access-h7dcp" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983047    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01d89188-b530-4373-9ea9-b58a4122c8cd" volumeName="kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983056    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983063    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983070    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983078    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66b60042-598e-49bb-8de0-009bd4c1bd6b" volumeName="kubernetes.io/empty-dir/66b60042-598e-49bb-8de0-009bd4c1bd6b-tmp-dir" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983085    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5e25b748-6ffe-488b-a210-24f1b4d1ce57" volumeName="kubernetes.io/secret/5e25b748-6ffe-488b-a210-24f1b4d1ce57-tls-certificates" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983109    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-federate-client-tls" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983116    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" volumeName="kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983124    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162c7603-5367-4d79-8ff6-23b05743cf35" volumeName="kubernetes.io/projected/162c7603-5367-4d79-8ff6-23b05743cf35-kube-api-access-6qms8" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983136    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f7acedd-ae54-4ad5-bef2-da836c8ef78e" volumeName="kubernetes.io/configmap/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-metrics-client-ca" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983145    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983153    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57d0d345-d190-4a7b-b61c-f2f2da64ffc8" volumeName="kubernetes.io/configmap/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-serviceca" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983162    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ebd1a13-21bc-4de8-aaad-4e289981ee10" volumeName="kubernetes.io/secret/6ebd1a13-21bc-4de8-aaad-4e289981ee10-cert" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983196    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983206    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7" seLinuxMountContext=""
Oct 30 00:39:17.983207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983213    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="57d0d345-d190-4a7b-b61c-f2f2da64ffc8" volumeName="kubernetes.io/projected/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-kube-api-access-zflwb" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983231    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d14ce64-f501-4c56-aeff-137f931bfea9" volumeName="kubernetes.io/secret/0d14ce64-f501-4c56-aeff-137f931bfea9-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983240    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983255    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636aa0c9-51c4-43d1-ad08-78922681e9d7" volumeName="kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983263    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2a1b468-478c-445d-af74-05dac392aec7" volumeName="kubernetes.io/projected/a2a1b468-478c-445d-af74-05dac392aec7-kube-api-access-dmcw7" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983270    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983278    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162c7603-5367-4d79-8ff6-23b05743cf35" volumeName="kubernetes.io/empty-dir/162c7603-5367-4d79-8ff6-23b05743cf35-volume-directive-shadow" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983287    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb95c197-005f-4deb-b563-107e7e0aff33" volumeName="kubernetes.io/empty-dir/fb95c197-005f-4deb-b563-107e7e0aff33-tmp-dir" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983294    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb95c197-005f-4deb-b563-107e7e0aff33" volumeName="kubernetes.io/secret/fb95c197-005f-4deb-b563-107e7e0aff33-metrics-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983301    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983315    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983325    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f7acedd-ae54-4ad5-bef2-da836c8ef78e" volumeName="kubernetes.io/projected/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-kube-api-access-clvh4" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983332    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="66b60042-598e-49bb-8de0-009bd4c1bd6b" volumeName="kubernetes.io/projected/66b60042-598e-49bb-8de0-009bd4c1bd6b-kube-api-access-ts7xc" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983343    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" volumeName="kubernetes.io/projected/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-kube-api-access-5h74d" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983351    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983359    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636aa0c9-51c4-43d1-ad08-78922681e9d7" volumeName="kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983366    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983373    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/empty-dir/4d53587f-0770-4a39-a369-dcc257c949f5-tmp" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983381    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01a7c97e-b248-4348-a07a-77652f9ce3c9" volumeName="kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-env-overrides" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983390    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983397    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983404    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47f11449-4b9d-44df-8fed-68f31ded1601" volumeName="kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983412    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" volumeName="kubernetes.io/empty-dir/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-textfile" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983420    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01d89188-b530-4373-9ea9-b58a4122c8cd" volumeName="kubernetes.io/projected/01d89188-b530-4373-9ea9-b58a4122c8cd-kube-api-access-ct2s6" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983427    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162c7603-5367-4d79-8ff6-23b05743cf35" volumeName="kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983435    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a1157b8-e332-4163-b20a-1edbbe85f786" volumeName="kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983442    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636aa0c9-51c4-43d1-ad08-78922681e9d7" volumeName="kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983450    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983457    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983465    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68a4bb84-29e4-43b6-92df-7a4ea03ee820" volumeName="kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-tuned" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983472    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2a1b468-478c-445d-af74-05dac392aec7" volumeName="kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983480    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-metrics-client-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983487    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983495    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47f11449-4b9d-44df-8fed-68f31ded1601" volumeName="kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983505    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5a3f42a4-9d45-4d56-952c-4dd04bb47261" volumeName="kubernetes.io/secret/5a3f42a4-9d45-4d56-952c-4dd04bb47261-agent-certs" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983513    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636aa0c9-51c4-43d1-ad08-78922681e9d7" volumeName="kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983521    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3" volumeName="kubernetes.io/projected/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kube-api-access-nsrkq" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983547    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983555    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2a1b468-478c-445d-af74-05dac392aec7" volumeName="kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983584    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" volumeName="kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983593    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-client-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983601    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01d89188-b530-4373-9ea9-b58a4122c8cd" volumeName="kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-client-ca-bundle" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983609    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983618    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162c7603-5367-4d79-8ff6-23b05743cf35" volumeName="kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983625    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f7acedd-ae54-4ad5-bef2-da836c8ef78e" volumeName="kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983632    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636aa0c9-51c4-43d1-ad08-78922681e9d7" volumeName="kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983640    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983647    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01d89188-b530-4373-9ea9-b58a4122c8cd" volumeName="kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983654    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983662    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983669    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/projected/4d53587f-0770-4a39-a369-dcc257c949f5-kube-api-access-4zhgq" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983676    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-grpc-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983684    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="5a3f42a4-9d45-4d56-952c-4dd04bb47261" volumeName="kubernetes.io/configmap/5a3f42a4-9d45-4d56-952c-4dd04bb47261-konnectivity-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983691    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b5d71931-979a-4170-abe4-d22055d75da2" volumeName="kubernetes.io/projected/b5d71931-979a-4170-abe4-d22055d75da2-kube-api-access-tvxcg" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983698    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" volumeName="kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-metrics-client-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983706    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01a7c97e-b248-4348-a07a-77652f9ce3c9" volumeName="kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983714    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01a7c97e-b248-4348-a07a-77652f9ce3c9" volumeName="kubernetes.io/projected/01a7c97e-b248-4348-a07a-77652f9ce3c9-kube-api-access-8zfkx" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983721    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983727    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c64e9e6d-c2b7-4520-b5d9-e8eb86720bab" volumeName="kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983734    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162c7603-5367-4d79-8ff6-23b05743cf35" volumeName="kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-metrics-client-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983742    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636aa0c9-51c4-43d1-ad08-78922681e9d7" volumeName="kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983749    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6ebd1a13-21bc-4de8-aaad-4e289981ee10" volumeName="kubernetes.io/projected/6ebd1a13-21bc-4de8-aaad-4e289981ee10-kube-api-access-ndnng" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983757    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb95c197-005f-4deb-b563-107e7e0aff33" volumeName="kubernetes.io/configmap/fb95c197-005f-4deb-b563-107e7e0aff33-config-volume" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983764    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983771    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b5d71931-979a-4170-abe4-d22055d75da2" volumeName="kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-cni-binary-copy" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983778    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" volumeName="kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983787    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fb95c197-005f-4deb-b563-107e7e0aff33" volumeName="kubernetes.io/projected/fb95c197-005f-4deb-b563-107e7e0aff33-kube-api-access-jpqk8" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983794    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01d89188-b530-4373-9ea9-b58a4122c8cd" volumeName="kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983801    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983808    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24b870b7-104e-44b7-b15a-8db848e34bfc" volumeName="kubernetes.io/projected/24b870b7-104e-44b7-b15a-8db848e34bfc-kube-api-access-bngtx" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983816    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2a1b468-478c-445d-af74-05dac392aec7" volumeName="kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-binary-copy" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983823    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/projected/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-kube-api-access-5497b" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983830    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983837    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0b672b37-e2e1-42f5-8adc-001a7ec77185" volumeName="kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983845    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983852    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983860    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983866    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="47f11449-4b9d-44df-8fed-68f31ded1601" volumeName="kubernetes.io/configmap/47f11449-4b9d-44df-8fed-68f31ded1601-metrics-client-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983873    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983882    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d14ce64-f501-4c56-aeff-137f931bfea9" volumeName="kubernetes.io/empty-dir/0d14ce64-f501-4c56-aeff-137f931bfea9-data-volume" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983889    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3a1157b8-e332-4163-b20a-1edbbe85f786" volumeName="kubernetes.io/projected/3a1157b8-e332-4163-b20a-1edbbe85f786-kube-api-access-7xzw8" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983896    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636aa0c9-51c4-43d1-ad08-78922681e9d7" volumeName="kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983903    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983910    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd83da39-c295-436b-a07c-4c41c68de6be" volumeName="kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983917    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01d89188-b530-4373-9ea9-b58a4122c8cd" volumeName="kubernetes.io/empty-dir/01d89188-b530-4373-9ea9-b58a4122c8cd-audit-log" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983926    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983935    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01a7c97e-b248-4348-a07a-77652f9ce3c9" volumeName="kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-config" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983942    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983949    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983958    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68a4bb84-29e4-43b6-92df-7a4ea03ee820" volumeName="kubernetes.io/projected/68a4bb84-29e4-43b6-92df-7a4ea03ee820-kube-api-access-h7qlk" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983965    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ae0fed46-f827-4a14-879c-b0c4a4fa7db8" volumeName="kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-serving-certs-ca-bundle" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983973    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0d14ce64-f501-4c56-aeff-137f931bfea9" volumeName="kubernetes.io/configmap/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983980    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="68a4bb84-29e4-43b6-92df-7a4ea03ee820" volumeName="kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-tmp" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983988    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="162c7603-5367-4d79-8ff6-23b05743cf35" volumeName="kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 00:39:17.983923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.983996    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" volumeName="kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984015    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24c36423-7a10-4a80-a3c8-2e3835c3d12a" volumeName="kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984023    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4d53587f-0770-4a39-a369-dcc257c949f5" volumeName="kubernetes.io/configmap/4d53587f-0770-4a39-a369-dcc257c949f5-metrics-client-ca" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984034    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b5d71931-979a-4170-abe4-d22055d75da2" volumeName="kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-multus-daemon-config" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984041    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01a7c97e-b248-4348-a07a-77652f9ce3c9" volumeName="kubernetes.io/secret/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984048    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="01d89188-b530-4373-9ea9-b58a4122c8cd" volumeName="kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984055    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f7acedd-ae54-4ad5-bef2-da836c8ef78e" volumeName="kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-tls" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984063    6011 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="24b870b7-104e-44b7-b15a-8db848e34bfc" volumeName="kubernetes.io/configmap/24b870b7-104e-44b7-b15a-8db848e34bfc-iptables-alerter-script" seLinuxMountContext=""
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984070    6011 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 00:39:17.988087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.984075    6011 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 00:39:17.989608 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.989586    6011 factory.go:55] Registering systemd factory
Oct 30 00:39:17.989689 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.989612    6011 factory.go:223] Registration of the systemd container factory successfully
Oct 30 00:39:17.990206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.989847    6011 factory.go:153] Registering CRI-O factory
Oct 30 00:39:17.990206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.989868    6011 factory.go:223] Registration of the crio container factory successfully
Oct 30 00:39:17.990206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.989931    6011 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 00:39:17.990206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.989962    6011 factory.go:103] Registering Raw factory
Oct 30 00:39:17.990206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.989978    6011 manager.go:1199] Started watching for new ooms in manager
Oct 30 00:39:17.990550 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:17.990536    6011 manager.go:319] Starting recovery of all containers
Oct 30 00:39:18.011455 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.011404    6011 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 00:39:18.013137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.012906    6011 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 00:39:18.013137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.012929    6011 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 00:39:18.013137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.012950    6011 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 00:39:18.013137 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:18.012988    6011 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 00:39:18.027356 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.027304    6011 generic.go:356] "Generic (PLEG): container finished" podID="3e516b0f6958377721da4db26ec47f8e" containerID="008008a36334aec87de5602740b5dd59f614014d70225c56318b41f40a62da89" exitCode=0
Oct 30 00:39:18.029866 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.029823    6011 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:39:18.030449 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.030432    6011 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 00:39:18.035884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.035852    6011 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:39:18.042333 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.042310    6011 generic.go:356] "Generic (PLEG): container finished" podID="f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc" containerID="4adccfc08ebac297ac449ffdc1a197af8a29a197544082196a7b25b754ae4c0b" exitCode=0
Oct 30 00:39:18.048960 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.048933    6011 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:39:18.049514 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.049496    6011 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 00:39:18.068228 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.068199    6011 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f" exitCode=0
Oct 30 00:39:18.095085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.095009    6011 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="0648bbe07e2f1fe8df135eaead88e8c8e7291cbde961cc9e72fdf7bdb4e14767" exitCode=0
Oct 30 00:39:18.095085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.095037    6011 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="7704e2f2d4cd45f0c716f6a0d1c7616a71e9e696b85654968401f543a1bb0b6b" exitCode=0
Oct 30 00:39:18.095085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.095050    6011 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="4098c8cd7475f842a973c8b4b153bc570fdbb1b408155fbe1c282738c76d80cb" exitCode=0
Oct 30 00:39:18.095085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.095061    6011 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="c5c5841ba070dedc1ec6fa9850f6e4df9ff42707a2fba98a8c3e029787f829f9" exitCode=0
Oct 30 00:39:18.095085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.095070    6011 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="d1e108341e402f7270013ca2f166de6c7b8909b2955d067fbcb4874193145e29" exitCode=0
Oct 30 00:39:18.095085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.095078    6011 generic.go:356] "Generic (PLEG): container finished" podID="a2a1b468-478c-445d-af74-05dac392aec7" containerID="c062b6a4821c0d56074e2f3bbacfb7c922724255567a84b9fdaf02c2941f040c" exitCode=0
Oct 30 00:39:18.113060 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:18.113030    6011 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 00:39:18.129212 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.129061    6011 manager.go:324] Recovery completed
Oct 30 00:39:18.165012 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.164987    6011 cpu_manager.go:226] "Starting" policy="none"
Oct 30 00:39:18.165012 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165005    6011 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165022    6011 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165183    6011 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165196    6011 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165214    6011 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165219    6011 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165225    6011 policy_none.go:50] "Start"
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165233    6011 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 00:39:18.165245 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165242    6011 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 00:39:18.165500 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165366    6011 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 00:39:18.165500 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165373    6011 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 00:39:18.165500 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.165378    6011 policy_none.go:44] "Start"
Oct 30 00:39:18.171141 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.171107    6011 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 00:39:18.176226 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.174988    6011 manager.go:341] "Starting Device Plugin manager"
Oct 30 00:39:18.176479 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:18.175022    6011 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 00:39:18.176557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.176482    6011 server.go:87] "Starting device plugin registration server"
Oct 30 00:39:18.176787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.176770    6011 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 00:39:18.176911 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.176786    6011 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 00:39:18.177014 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.176996    6011 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 00:39:18.177211 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.177118    6011 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 00:39:18.177211 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.177194    6011 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 00:39:18.177211 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.177206    6011 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 00:39:18.177484 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.177345    6011 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T00:39:18.177158126Z","UUID":"d5c8949e-4c28-4070-af63-9e757d20b87d","Handler":null,"Name":"","Endpoint":""}
Oct 30 00:39:18.179567 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:18.179537    6011 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 00:39:18.180301 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.180279    6011 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 00:39:18.180391 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.180315    6011 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 00:39:18.277843 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.277811    6011 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 00:39:18.280963 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.280933    6011 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientMemory"
Oct 30 00:39:18.280963 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.280963    6011 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasNoDiskPressure"
Oct 30 00:39:18.281100 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.280974    6011 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-131-254.ec2.internal" event="NodeHasSufficientPID"
Oct 30 00:39:18.281100 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.281053    6011 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.313151 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.313113    6011 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"]
Oct 30 00:39:18.313753 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.313725    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal" event={"ID":"7a3c183736bfb0e3d8475eab25325f35","Type":"ContainerStarted","Data":"eba591903716040e75bb9b7d66c5513098ce878f2e97c368f8f8792e4322da5f"}
Oct 30 00:39:18.313811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.313759    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal" event={"ID":"7a3c183736bfb0e3d8475eab25325f35","Type":"ContainerStarted","Data":"e1696374dc8343e250e7d53592dce065d35b19043f881adc5eb2ddbc4674ccf3"}
Oct 30 00:39:18.313811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.313773    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal" event={"ID":"3e516b0f6958377721da4db26ec47f8e","Type":"ContainerStarted","Data":"b9ea781b817e79e507d5a5c0170e0dfc6026d0ef69eee03ac611f219e838002b"}
Oct 30 00:39:18.313811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.313784    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal" event={"ID":"3e516b0f6958377721da4db26ec47f8e","Type":"ContainerDied","Data":"008008a36334aec87de5602740b5dd59f614014d70225c56318b41f40a62da89"}
Oct 30 00:39:18.313811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.313799    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal" event={"ID":"3e516b0f6958377721da4db26ec47f8e","Type":"ContainerStarted","Data":"4fd67a124d5a078f39aaf704309b098b79768c355163637e762928323e66d7cc"}
Oct 30 00:39:18.313926 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.313784    6011 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.314023 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.314010    6011 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.317183 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.317158    6011 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.317254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.317217    6011 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.338347 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.338326    6011 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:39:18.338435 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:18.338360    6011 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-131-254.ec2.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.338821 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.338807    6011 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:39:18.338856 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:18.338842    6011 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.432584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.432494    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a3c183736bfb0e3d8475eab25325f35-config\") pod \"kube-apiserver-proxy-ip-10-0-131-254.ec2.internal\" (UID: \"7a3c183736bfb0e3d8475eab25325f35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.432584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.432531    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.432584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.432557    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.533630 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.533591    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.533630 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.533607    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.533860 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.533642    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a3c183736bfb0e3d8475eab25325f35-config\") pod \"kube-apiserver-proxy-ip-10-0-131-254.ec2.internal\" (UID: \"7a3c183736bfb0e3d8475eab25325f35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.533860 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.533661    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.533860 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.533679    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a3c183736bfb0e3d8475eab25325f35-config\") pod \"kube-apiserver-proxy-ip-10-0-131-254.ec2.internal\" (UID: \"7a3c183736bfb0e3d8475eab25325f35\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.533860 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.533698    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e516b0f6958377721da4db26ec47f8e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" (UID: \"3e516b0f6958377721da4db26ec47f8e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:18.963035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:18.963002    6011 apiserver.go:52] "Watching apiserver"
Oct 30 00:39:19.017128 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.017097    6011 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 00:39:19.017830 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.017806    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-ql49r","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2","openshift-network-operator/iptables-alerter-m6vlx","openshift-image-registry/node-ca-g6srk","openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp","openshift-monitoring/alertmanager-main-0","openshift-console/console-59ff97898d-4tlzq","openshift-insights/insights-runtime-extractor-drhmz","openshift-cluster-node-tuning-operator/tuned-5sv2w","kube-system/global-pull-secret-syncer-x2s8q","openshift-console/console-78cf5d5fdb-fxvv5","openshift-monitoring/openshift-state-metrics-7bd7489957-84864","openshift-console/console-64684dbfbb-flsst","kube-system/kube-apiserver-proxy-ip-10-0-131-254.ec2.internal","openshift-monitoring/metrics-server-5dc88c6bb9-9zp98","openshift-multus/multus-additional-cni-plugins-ckp4t","openshift-ovn-kubernetes/ovnkube-node-vwlnt","openshift-monitoring/node-exporter-pztnw","openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal","openshift-dns/dns-default-qlw4m","openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv","openshift-network-diagnostics/network-check-target-pgl9q","openshift-multus/network-metrics-daemon-52dwv","kube-system/konnectivity-agent-p8fxn","openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs","openshift-multus/multus-ddf5d","openshift-ingress-canary/ingress-canary-7zp52","openshift-monitoring/telemeter-client-6959d644bc-xfsth"]
Oct 30 00:39:19.033477 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.033453    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.034406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034388    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-9xms5\""
Oct 30 00:39:19.034605 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034591    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2bp5j\""
Oct 30 00:39:19.034693 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034604    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.034693 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034625    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.034693 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034664    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 00:39:19.034693 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034666    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kg5gj\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034690    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034697    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034716    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034807    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034815    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034807    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034824    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5s9q7\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034849    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 00:39:19.034854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034855    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.035124 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034872    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 00:39:19.035124 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.034887    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-44cn9\""
Oct 30 00:39:19.045261 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.045239    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.046574 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.046557    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.046642 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.046574    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.046688 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.046654    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-tmln4\""
Oct 30 00:39:19.052659 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052642    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 00:39:19.052911 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052896    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 00:39:19.052973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052902    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 00:39:19.052973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052924    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 00:39:19.052973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052933    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 00:39:19.052973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052933    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 00:39:19.052973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052954    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Oct 30 00:39:19.052973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052963    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 00:39:19.052973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052964    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 00:39:19.053256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.052986    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 00:39:19.053256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053089    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 00:39:19.053256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053210    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Oct 30 00:39:19.053256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053229    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 00:39:19.053256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053249    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-qwk9j\""
Oct 30 00:39:19.053432 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053261    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 00:39:19.053432 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053277    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-crwd9\""
Oct 30 00:39:19.053432 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053255    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 00:39:19.053432 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053277    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 00:39:19.053432 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053291    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7gr5j\""
Oct 30 00:39:19.053670 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053655    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p7bx4\""
Oct 30 00:39:19.053899 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053806    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 00:39:19.053977 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053916    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.053977 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.053916    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 00:39:19.054201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054166    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 00:39:19.054267 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054220    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.054337 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054310    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 00:39:19.054337 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054319    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 00:39:19.054337 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054324    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-w7tqr\""
Oct 30 00:39:19.054630 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054498    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 00:39:19.054630 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054499    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 00:39:19.054630 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054521    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 00:39:19.054816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054725    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 00:39:19.054816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054742    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 00:39:19.054816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054750    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.054816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054811    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.054948 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.054911    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.055150 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055135    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bqzrd\""
Oct 30 00:39:19.055256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055168    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-xt47p\""
Oct 30 00:39:19.055340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055326    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lpt8z\""
Oct 30 00:39:19.055387 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055370    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 00:39:19.055444 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055398    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-16hl7253o93j0\""
Oct 30 00:39:19.055444 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055430    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.055501 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055444    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.055501 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055469    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 00:39:19.055501 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055483    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 00:39:19.055623 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055520    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 00:39:19.055623 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055524    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 00:39:19.055623 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055614    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 00:39:19.055748 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055644    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.055798 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055785    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 00:39:19.055798 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055794    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.055971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055794    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 00:39:19.055971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055832    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.055971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055870    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.055971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055898    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 00:39:19.055971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.055926    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 00:39:19.056166 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056058    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 00:39:19.056331 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056312    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 00:39:19.056398 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056333    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ptmsd\""
Oct 30 00:39:19.056398 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056337    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ckcc7\""
Oct 30 00:39:19.056398 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056316    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-545pj\""
Oct 30 00:39:19.056398 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056340    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 00:39:19.056398 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056359    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 00:39:19.056574 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056411    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 00:39:19.056574 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.056437    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-jxbz5\""
Oct 30 00:39:19.058709 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.058677    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 00:39:19.058709 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.058689    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-907sh4mbvn436\""
Oct 30 00:39:19.058709 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.058705    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hs87l\""
Oct 30 00:39:19.058903 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.058738    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-smcj2\""
Oct 30 00:39:19.064147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.064134    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 00:39:19.064242 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.064185    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k8zgn\""
Oct 30 00:39:19.064297 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.064221    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t5wsq\""
Oct 30 00:39:19.065786 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.065772    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 00:39:19.082796 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.082767    6011 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 00:39:19.104908 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.104889    6011 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:19.126945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.126917    6011 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 00:39:19.127051 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:19.126979    6011 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-254.ec2.internal"
Oct 30 00:39:19.135280 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135261    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-serviceca\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:39:19.135388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135286    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.135388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135303    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.135388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135321    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-client-certs\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.135388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135342    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-registration-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.135388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135360    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b870b7-104e-44b7-b15a-8db848e34bfc-host-slash\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:39:19.135614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135540    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-conf-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.135614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135573    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-client-certs\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.135718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135637    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-serving-certs-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.135718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135678    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53587f-0770-4a39-a369-dcc257c949f5-tmp\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.135718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135711    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.135852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135743    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-systemd-units\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.135852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135771    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-slash\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.135852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135791    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.135852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135790    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4d53587f-0770-4a39-a369-dcc257c949f5-tmp\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.135852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135802    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135889    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f11449-4b9d-44df-8fed-68f31ded1601-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135919    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135936    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135971    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.135995    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136015    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-serviceca\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136023    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136029    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.136090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136089    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-root\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136123    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136122    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136254    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a3f42a4-9d45-4d56-952c-4dd04bb47261-agent-certs\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136254    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136270    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136421    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/47f11449-4b9d-44df-8fed-68f31ded1601-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136443    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136475    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-metrics-server-audit-profiles\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136502    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-tls\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.136533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136501    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5a3f42a4-9d45-4d56-952c-4dd04bb47261-agent-certs\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136606    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136748    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-metrics-server-audit-profiles\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136757    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136764    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136788    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136800    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-secret-metrics-server-tls\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136820    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136847    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb95c197-005f-4deb-b563-107e7e0aff33-metrics-tls\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136866    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136875    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136912    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7dcp\" (UniqueName: \"kubernetes.io/projected/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-api-access-h7dcp\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136940    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136964    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.136983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136988    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-tls\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.136995    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137014    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137046    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-socket-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137073    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-kubernetes\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137099    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-kubelet\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137130    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5497b\" (UniqueName: \"kubernetes.io/projected/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-kube-api-access-5497b\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137155    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zfkx\" (UniqueName: \"kubernetes.io/projected/01a7c97e-b248-4348-a07a-77652f9ce3c9-kube-api-access-8zfkx\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137165    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137013    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137250    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137255    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-tls\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137275    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-sys\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137392    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a3f42a4-9d45-4d56-952c-4dd04bb47261-konnectivity-ca\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137459    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d53587f-0770-4a39-a369-dcc257c949f5-metrics-client-ca\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137478    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/162c7603-5367-4d79-8ff6-23b05743cf35-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137490    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137525    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-textfile\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137543    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137554    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-run\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.137643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137591    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-cni-binary-copy\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137671    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5a3f42a4-9d45-4d56-952c-4dd04bb47261-konnectivity-ca\") pod \"konnectivity-agent-p8fxn\" (UID: \"5a3f42a4-9d45-4d56-952c-4dd04bb47261\") " pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137675    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-textfile\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137708    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a1157b8-e332-4163-b20a-1edbbe85f786-metrics-certs\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137708    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-hostroot\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137751    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137761    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-cni-binary-copy\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137775    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137782    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137802    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137911    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137927    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4d53587f-0770-4a39-a369-dcc257c949f5-metrics-client-ca\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137941    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137950    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.137978    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138013    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zflwb\" (UniqueName: \"kubernetes.io/projected/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-kube-api-access-zflwb\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138019    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fb95c197-005f-4deb-b563-107e7e0aff33-metrics-tls\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138058    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwlfj\" (UniqueName: \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138080    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138088    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-host\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138121    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-cnibin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138164    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-env-overrides\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138169    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138242    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138274    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95rd7\" (UniqueName: \"kubernetes.io/projected/47f11449-4b9d-44df-8fed-68f31ded1601-kube-api-access-95rd7\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138341    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-sys-fs\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138368    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138367    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d14ce64-f501-4c56-aeff-137f931bfea9-data-volume\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138426    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d14ce64-f501-4c56-aeff-137f931bfea9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138451    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0d14ce64-f501-4c56-aeff-137f931bfea9-data-volume\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138466    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138490    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138453    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-env-overrides\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138500    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/162c7603-5367-4d79-8ff6-23b05743cf35-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138553    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/162c7603-5367-4d79-8ff6-23b05743cf35-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138552    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138603    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138620    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-multus-certs\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138658    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-systemd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138686    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmtz7\" (UniqueName: \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138706    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.138760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138737    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138900    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138910    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138932    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-systemd\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138957    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d14ce64-f501-4c56-aeff-137f931bfea9-crio-socket\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138785    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-binary-copy\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138983    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.138995    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139019    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01d89188-b530-4373-9ea9-b58a4122c8cd-audit-log\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139063    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-tuned\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139202    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-tuned\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139262    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/01d89188-b530-4373-9ea9-b58a4122c8cd-audit-log\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139271    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139292    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/01d89188-b530-4373-9ea9-b58a4122c8cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139331    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139355    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-etc-kubernetes\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139445    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139503    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139539    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zhgq\" (UniqueName: \"kubernetes.io/projected/4d53587f-0770-4a39-a369-dcc257c949f5-kube-api-access-4zhgq\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139567    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-netns\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139596    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139623    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139650    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-multus-daemon-config\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139689    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8229s\" (UniqueName: \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139729    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-ovn\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139739    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139739    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139830    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-bin\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139884    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139908    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139913    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b5d71931-979a-4170-abe4-d22055d75da2-multus-daemon-config\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139947    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139953    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.139974    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140011    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140105    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140119    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/47f11449-4b9d-44df-8fed-68f31ded1601-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140140    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140209    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140354    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-metrics-client-ca\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140396    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsrkq\" (UniqueName: \"kubernetes.io/projected/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kube-api-access-nsrkq\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140417    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140422    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162c7603-5367-4d79-8ff6-23b05743cf35-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140433    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140454    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-metrics-client-ca\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140422    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140544    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-script-lib\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140553    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140515    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-q2s6t\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140582    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5nmrp\""
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140653    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e25b748-6ffe-488b-a210-24f1b4d1ce57-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-5tklp\" (UID: \"5e25b748-6ffe-488b-a210-24f1b4d1ce57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:39:19.140666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140667    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-m2pgd\""
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140757    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140762    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140671    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-host\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140813    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140841    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-system-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140877    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-script-lib\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.140964    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141039    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5e25b748-6ffe-488b-a210-24f1b4d1ce57-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-5tklp\" (UID: \"5e25b748-6ffe-488b-a210-24f1b4d1ce57\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141038    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141162    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141231    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-socket-dir-parent\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141267    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvxcg\" (UniqueName: \"kubernetes.io/projected/b5d71931-979a-4170-abe4-d22055d75da2-kube-api-access-tvxcg\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141294    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-var-lib-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141304    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141318    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-netd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141337    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141353    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovn-node-metrics-cert\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141390    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-tmp\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141456    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a4bb84-29e4-43b6-92df-7a4ea03ee820-tmp\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141496    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7qlk\" (UniqueName: \"kubernetes.io/projected/68a4bb84-29e4-43b6-92df-7a4ea03ee820-kube-api-access-h7qlk\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141545    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bngtx\" (UniqueName: \"kubernetes.io/projected/24b870b7-104e-44b7-b15a-8db848e34bfc-kube-api-access-bngtx\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141569    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141595    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141664    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovn-node-metrics-cert\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141668    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141684    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141690    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141690    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141738    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmcw7\" (UniqueName: \"kubernetes.io/projected/a2a1b468-478c-445d-af74-05dac392aec7-kube-api-access-dmcw7\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141776    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141810    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clvh4\" (UniqueName: \"kubernetes.io/projected/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-kube-api-access-clvh4\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141838    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h74d\" (UniqueName: \"kubernetes.io/projected/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-kube-api-access-5h74d\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141870    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141880    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.141915    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysconfig\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142011    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-grpc-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142045    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142101    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142124    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b870b7-104e-44b7-b15a-8db848e34bfc-iptables-alerter-script\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142129    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142145    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xzw8\" (UniqueName: \"kubernetes.io/projected/3a1157b8-e332-4163-b20a-1edbbe85f786-kube-api-access-7xzw8\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142190    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142258    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142272    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-accelerators-collector-config\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142331    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/24b870b7-104e-44b7-b15a-8db848e34bfc-iptables-alerter-script\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142338    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-grpc-tls\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142359    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142385    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142403    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142428    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-cnibin\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142542    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-device-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142574    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142593    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142597    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142621    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142623    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142687    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-log-socket\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142717    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fn4jb\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142744    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-client-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142782    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66b60042-598e-49bb-8de0-009bd4c1bd6b-hosts-file\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142800    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142843    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b60042-598e-49bb-8de0-009bd4c1bd6b-tmp-dir\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142856    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-original-pull-secret\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142893    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-modprobe-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142921    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/66b60042-598e-49bb-8de0-009bd4c1bd6b-tmp-dir\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:39:19.142923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.142954    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpqk8\" (UniqueName: \"kubernetes.io/projected/fb95c197-005f-4deb-b563-107e7e0aff33-kube-api-access-jpqk8\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143001    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-multus\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143027    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/01d89188-b530-4373-9ea9-b58a4122c8cd-client-ca-bundle\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143085    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qms8\" (UniqueName: \"kubernetes.io/projected/162c7603-5367-4d79-8ff6-23b05743cf35-kube-api-access-6qms8\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143132    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143165    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb95c197-005f-4deb-b563-107e7e0aff33-tmp-dir\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143225    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-kubelet-config\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143243    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-bin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143268    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-kubelet\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143297    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fb95c197-005f-4deb-b563-107e7e0aff33-tmp-dir\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143310    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-config\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143338    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143455    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-var-lib-kubelet\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143485    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/01a7c97e-b248-4348-a07a-77652f9ce3c9-ovnkube-config\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143558    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-k8s-cni-cncf-io\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143593    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143618    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143641    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-sys\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143658    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb95c197-005f-4deb-b563-107e7e0aff33-config-volume\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143690    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143721    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143807    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ebd1a13-21bc-4de8-aaad-4e289981ee10-cert\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143830    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fb95c197-005f-4deb-b563-107e7e0aff33-config-volume\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143837    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143864    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-wtmp\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143897    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143909    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143936    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts7xc\" (UniqueName: \"kubernetes.io/projected/66b60042-598e-49bb-8de0-009bd4c1bd6b-kube-api-access-ts7xc\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143947    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143961    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-dbus\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.143985    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6ebd1a13-21bc-4de8-aaad-4e289981ee10-cert\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144031    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-netns\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144050    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144084    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-system-cni-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144097    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a2a1b468-478c-445d-af74-05dac392aec7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144141    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct2s6\" (UniqueName: \"kubernetes.io/projected/01d89188-b530-4373-9ea9-b58a4122c8cd-kube-api-access-ct2s6\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144162    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-conf\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144204    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-lib-modules\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144239    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-node-log\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144275    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144319    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144419    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144445    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144456    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144483    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-federate-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144500    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144510    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-metrics-client-ca\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144545    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144644    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-etc-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144697    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-federate-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144751    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndnng\" (UniqueName: \"kubernetes.io/projected/6ebd1a13-21bc-4de8-aaad-4e289981ee10-kube-api-access-ndnng\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144767    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-telemeter-client-tls\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144801    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-metrics-client-ca\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.145427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.144911    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.146944 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.145991    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-serving-certs-ca-bundle\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.146944 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.146033    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-secret-telemeter-client\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.146944 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.146033    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.147231 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.147152    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.147231 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.147155    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.148309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.148293    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.149550 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.149534    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0d14ce64-f501-4c56-aeff-137f931bfea9-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.149914 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.149898    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.150034 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.150022    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.150034 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.150029    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4d53587f-0770-4a39-a369-dcc257c949f5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.163508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.163487    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bngtx\" (UniqueName: \"kubernetes.io/projected/24b870b7-104e-44b7-b15a-8db848e34bfc-kube-api-access-bngtx\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:39:19.163806 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.163790    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7qlk\" (UniqueName: \"kubernetes.io/projected/68a4bb84-29e4-43b6-92df-7a4ea03ee820-kube-api-access-h7qlk\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.163844 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.163819    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95rd7\" (UniqueName: \"kubernetes.io/projected/47f11449-4b9d-44df-8fed-68f31ded1601-kube-api-access-95rd7\") pod \"openshift-state-metrics-7bd7489957-84864\" (UID: \"47f11449-4b9d-44df-8fed-68f31ded1601\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-84864"
Oct 30 00:39:19.163939 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.163921    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsrkq\" (UniqueName: \"kubernetes.io/projected/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kube-api-access-nsrkq\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.163982 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.163958    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8229s\" (UniqueName: \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\") pod \"console-78cf5d5fdb-fxvv5\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") " pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:19.164708 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.164692    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zfkx\" (UniqueName: \"kubernetes.io/projected/01a7c97e-b248-4348-a07a-77652f9ce3c9-kube-api-access-8zfkx\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.164789 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.164770    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zflwb\" (UniqueName: \"kubernetes.io/projected/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-kube-api-access-zflwb\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:39:19.165815 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.165797    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5497b\" (UniqueName: \"kubernetes.io/projected/ae0fed46-f827-4a14-879c-b0c4a4fa7db8-kube-api-access-5497b\") pod \"telemeter-client-6959d644bc-xfsth\" (UID: \"ae0fed46-f827-4a14-879c-b0c4a4fa7db8\") " pod="openshift-monitoring/telemeter-client-6959d644bc-xfsth"
Oct 30 00:39:19.166357 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.166326    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmcw7\" (UniqueName: \"kubernetes.io/projected/a2a1b468-478c-445d-af74-05dac392aec7-kube-api-access-dmcw7\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.166949 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.166926    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zhgq\" (UniqueName: \"kubernetes.io/projected/4d53587f-0770-4a39-a369-dcc257c949f5-kube-api-access-4zhgq\") pod \"thanos-querier-64dbd87c4d-n7xkv\" (UID: \"4d53587f-0770-4a39-a369-dcc257c949f5\") " pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:19.167101 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.167071    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvxcg\" (UniqueName: \"kubernetes.io/projected/b5d71931-979a-4170-abe4-d22055d75da2-kube-api-access-tvxcg\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.167414 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.167395    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h74d\" (UniqueName: \"kubernetes.io/projected/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-kube-api-access-5h74d\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.167502 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.167479    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmtz7\" (UniqueName: \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\") pod \"console-59ff97898d-4tlzq\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") " pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:19.167569 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.167547    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpqk8\" (UniqueName: \"kubernetes.io/projected/fb95c197-005f-4deb-b563-107e7e0aff33-kube-api-access-jpqk8\") pod \"dns-default-qlw4m\" (UID: \"fb95c197-005f-4deb-b563-107e7e0aff33\") " pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:19.167635 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.167613    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7dcp\" (UniqueName: \"kubernetes.io/projected/0d14ce64-f501-4c56-aeff-137f931bfea9-kube-api-access-h7dcp\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.167695 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.167642    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xzw8\" (UniqueName: \"kubernetes.io/projected/3a1157b8-e332-4163-b20a-1edbbe85f786-kube-api-access-7xzw8\") pod \"network-metrics-daemon-52dwv\" (UID: \"3a1157b8-e332-4163-b20a-1edbbe85f786\") " pod="openshift-multus/network-metrics-daemon-52dwv"
Oct 30 00:39:19.167999 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.167980    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clvh4\" (UniqueName: \"kubernetes.io/projected/1f7acedd-ae54-4ad5-bef2-da836c8ef78e-kube-api-access-clvh4\") pod \"prometheus-operator-9cd89b74c-kn2fs\" (UID: \"1f7acedd-ae54-4ad5-bef2-da836c8ef78e\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-kn2fs"
Oct 30 00:39:19.168291 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.168270    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fn4jb\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\") pod \"alertmanager-main-0\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:39:19.168372 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.168276    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndnng\" (UniqueName: \"kubernetes.io/projected/6ebd1a13-21bc-4de8-aaad-4e289981ee10-kube-api-access-ndnng\") pod \"ingress-canary-7zp52\" (UID: \"6ebd1a13-21bc-4de8-aaad-4e289981ee10\") " pod="openshift-ingress-canary/ingress-canary-7zp52"
Oct 30 00:39:19.168473 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.168453    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwlfj\" (UniqueName: \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\") pod \"console-64684dbfbb-flsst\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") " pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:19.168531 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.168460    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qms8\" (UniqueName: \"kubernetes.io/projected/162c7603-5367-4d79-8ff6-23b05743cf35-kube-api-access-6qms8\") pod \"kube-state-metrics-5dc8d7d89b-zbc6h\" (UID: \"162c7603-5367-4d79-8ff6-23b05743cf35\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-zbc6h"
Oct 30 00:39:19.168647 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.168629    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct2s6\" (UniqueName: \"kubernetes.io/projected/01d89188-b530-4373-9ea9-b58a4122c8cd-kube-api-access-ct2s6\") pod \"metrics-server-5dc88c6bb9-9zp98\" (UID: \"01d89188-b530-4373-9ea9-b58a4122c8cd\") " pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:19.168883 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.168867    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc4fj\" (UniqueName: \"kubernetes.io/projected/cd83da39-c295-436b-a07c-4c41c68de6be-kube-api-access-bc4fj\") pod \"network-check-target-pgl9q\" (UID: \"cd83da39-c295-436b-a07c-4c41c68de6be\") " pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:19.169035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.169015    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts7xc\" (UniqueName: \"kubernetes.io/projected/66b60042-598e-49bb-8de0-009bd4c1bd6b-kube-api-access-ts7xc\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:39:19.245791 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245691    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-host\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.245791 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245738    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-cnibin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.245791 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245765    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-sys-fs\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245823    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-multus-certs\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245836    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-host\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245849    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-systemd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245878    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-cnibin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245892    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-systemd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245895    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-sys-fs\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245888    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-multus-certs\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245952    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-systemd\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.245981    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d14ce64-f501-4c56-aeff-137f931bfea9-crio-socket\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246014    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246039    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-etc-kubernetes\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246066    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-netns\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246076    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0d14ce64-f501-4c56-aeff-137f931bfea9-crio-socket\") pod \"insights-runtime-extractor-drhmz\" (UID: \"0d14ce64-f501-4c56-aeff-137f931bfea9\") " pod="openshift-insights/insights-runtime-extractor-drhmz"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246091    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-ovn\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246282    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-systemd\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246285    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-bin\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246309    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-ovn\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246321    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-bin\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246327    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-host\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246351    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57d0d345-d190-4a7b-b61c-f2f2da64ffc8-host\") pod \"node-ca-g6srk\" (UID: \"57d0d345-d190-4a7b-b61c-f2f2da64ffc8\") " pod="openshift-image-registry/node-ca-g6srk"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246362    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-etc-kubernetes\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246374    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-netns\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246380    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246403    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-system-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246423    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246441    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-socket-dir-parent\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246468    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-var-lib-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246490    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-system-cni-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.246494 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246494    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-netd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246528    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-cni-netd\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246529    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-socket-dir-parent\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246536    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-var-lib-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246529    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246536    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246572    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246590    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysconfig\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246620    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-cnibin\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246630    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysconfig\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246636    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-device-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246656    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-cnibin\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246692    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-device-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246652    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-log-socket\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246723    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66b60042-598e-49bb-8de0-009bd4c1bd6b-hosts-file\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246746    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-modprobe-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246771    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-multus\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246749    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-log-socket\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246786    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/66b60042-598e-49bb-8de0-009bd4c1bd6b-hosts-file\") pod \"node-resolver-ql49r\" (UID: \"66b60042-598e-49bb-8de0-009bd4c1bd6b\") " pod="openshift-dns/node-resolver-ql49r"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246795    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246819    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-kubelet-config\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246841    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-modprobe-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246842    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-bin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246855    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-multus\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246860    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246876    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-kubelet\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246861    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-cni-bin\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246898    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-kubelet-config\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246910    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-kubelet\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246918    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-var-lib-kubelet\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246946    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-k8s-cni-cncf-io\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246974    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-sys\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.246985    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-var-lib-kubelet\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247000    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-k8s-cni-cncf-io\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247002    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-wtmp\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247040    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-dbus\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247052    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-sys\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247060    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-node-exporter-wtmp\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247066    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-netns\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247075    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c64e9e6d-c2b7-4520-b5d9-e8eb86720bab-dbus\") pod \"global-pull-secret-syncer-x2s8q\" (UID: \"c64e9e6d-c2b7-4520-b5d9-e8eb86720bab\") " pod="kube-system/global-pull-secret-syncer-x2s8q"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247101    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-run-netns\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247109    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247129    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-run-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247136    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-system-cni-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247158    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-conf\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247197    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-lib-modules\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.247168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247208    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a2a1b468-478c-445d-af74-05dac392aec7-system-cni-dir\") pod \"multus-additional-cni-plugins-ckp4t\" (UID: \"a2a1b468-478c-445d-af74-05dac392aec7\") " pod="openshift-multus/multus-additional-cni-plugins-ckp4t"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247223    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-node-log\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247252    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-etc-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247278    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-registration-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247300    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b870b7-104e-44b7-b15a-8db848e34bfc-host-slash\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247324    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-conf-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247330    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-node-log\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247340    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-lib-modules\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247347    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-systemd-units\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247370    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-slash\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247349    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-etc-openvswitch\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247379    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/24b870b7-104e-44b7-b15a-8db848e34bfc-host-slash\") pod \"iptables-alerter-m6vlx\" (UID: \"24b870b7-104e-44b7-b15a-8db848e34bfc\") " pod="openshift-network-operator/iptables-alerter-m6vlx"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247384    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-multus-conf-dir\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247391    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-root\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247423    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-systemd-units\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247427    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-slash\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247426    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-root\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247432    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247466    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-registration-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247477    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-socket-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247468    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-conf\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247493    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-kubernetes\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247514    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/01a7c97e-b248-4348-a07a-77652f9ce3c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-vwlnt\" (UID: \"01a7c97e-b248-4348-a07a-77652f9ce3c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247530    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-socket-dir\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247542    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-kubelet\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247571    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-sys\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247585    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-kubernetes\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247612    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-host-var-lib-kubelet\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247596    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-run\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247624    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc-sys\") pod \"node-exporter-pztnw\" (UID: \"f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc\") " pod="openshift-monitoring/node-exporter-pztnw"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247645    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-hostroot\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247667    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247707    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247706    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-run\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247739    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b5d71931-979a-4170-abe4-d22055d75da2-hostroot\") pod \"multus-ddf5d\" (UID: \"b5d71931-979a-4170-abe4-d22055d75da2\") " pod="openshift-multus/multus-ddf5d"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247748    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpgb2\" (UID: \"8f8e2990-d9f0-41a8-a8d6-09c67b54e9f3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpgb2"
Oct 30 00:39:19.248745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.247842    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/68a4bb84-29e4-43b6-92df-7a4ea03ee820-etc-sysctl-d\") pod \"tuned-5sv2w\" (UID: \"68a4bb84-29e4-43b6-92df-7a4ea03ee820\") " pod="openshift-cluster-node-tuning-operator/tuned-5sv2w"
Oct 30 00:39:19.333776 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:19.333749    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:20.111664 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:20.111627    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf5d5fdb-fxvv5" event={"ID":"0b672b37-e2e1-42f5-8adc-001a7ec77185","Type":"ContainerStarted","Data":"ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7"}
Oct 30 00:39:20.112225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:20.111684    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf5d5fdb-fxvv5" event={"ID":"0b672b37-e2e1-42f5-8adc-001a7ec77185","Type":"ContainerStarted","Data":"db67bdbb04cc28a261d52554b33192b8da20975fe2ef2cf082ea275dcaca59a0"}
Oct 30 00:39:20.702161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:20.702114    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:21.124261 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:21.124232    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qlw4m"
Oct 30 00:39:21.744026 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:21.744000    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-64dbd87c4d-n7xkv"
Oct 30 00:39:23.113026 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:23.112992    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:24.549822 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:24.549794    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-5tklp"
Oct 30 00:39:25.085211 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:25.085162    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:25.089841 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:25.089815    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:25.136342 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:25.136315    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:39:25.202433 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:25.202390    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64684dbfbb-flsst"]
Oct 30 00:39:26.453463 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:26.453425    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:26.453830 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:26.453694    6011 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 00:39:26.996602 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:26.996575    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:27.909995 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:27.909914    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:39:29.282187 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:29.282135    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:32.808668 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:32.808637    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vwlnt"
Oct 30 00:39:35.747696 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:35.747648    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:37.122800 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:37.122769    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:43.897867 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:43.897837    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p8fxn"
Oct 30 00:39:44.052201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.052137    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-59ff97898d-4tlzq" podUID="24c36423-7a10-4a80-a3c8-2e3835c3d12a" containerName="console" containerID="cri-o://3b999128fdbb7f9fe304379e317093d6be3fcbab8c5ba5df4074a529b290498b" gracePeriod=15
Oct 30 00:39:44.205380 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.205342    6011 generic.go:356] "Generic (PLEG): container finished" podID="24c36423-7a10-4a80-a3c8-2e3835c3d12a" containerID="3b999128fdbb7f9fe304379e317093d6be3fcbab8c5ba5df4074a529b290498b" exitCode=0
Oct 30 00:39:44.205550 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.205412    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff97898d-4tlzq" event={"ID":"24c36423-7a10-4a80-a3c8-2e3835c3d12a","Type":"ContainerDied","Data":"3b999128fdbb7f9fe304379e317093d6be3fcbab8c5ba5df4074a529b290498b"}
Oct 30 00:39:44.299870 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.299839    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:44.349457 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349424    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.349457 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349463    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.349705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349489    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\" (UniqueName: \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.349705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349517    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.349705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349559    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.349705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349590    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.349705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349643    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.349705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349679    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\") pod \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\" (UID: \"24c36423-7a10-4a80-a3c8-2e3835c3d12a\") "
Oct 30 00:39:44.350022 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.349894    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:39:44.350491 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.350420    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:39:44.350491 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.350411    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:39:44.350709 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.350518    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:39:44.350709 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.350543    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:39:44.352223 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.352155    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:39:44.352313 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.352217    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "kube-api-access-lmtz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:39:44.352313 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.352247    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert" pod "24c36423-7a10-4a80-a3c8-2e3835c3d12a" (UID: "24c36423-7a10-4a80-a3c8-2e3835c3d12a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450528    6011 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-trusted-ca-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450559    6011 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-serving-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450570    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmtz7\" (UniqueName: \"kubernetes.io/projected/24c36423-7a10-4a80-a3c8-2e3835c3d12a-kube-api-access-lmtz7\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450578    6011 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-oauth-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450589    6011 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-console-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450598    6011 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-oauth-serving-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450607    6011 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24c36423-7a10-4a80-a3c8-2e3835c3d12a-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:44.450611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:44.450617    6011 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24c36423-7a10-4a80-a3c8-2e3835c3d12a-tmp\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:45.210787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:45.210754    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59ff97898d-4tlzq" event={"ID":"24c36423-7a10-4a80-a3c8-2e3835c3d12a","Type":"ContainerDied","Data":"6f7b32e0fb17a75580847afe8e6f7b66a5350509354cf857e68e12ddb4578e4e"}
Oct 30 00:39:45.210787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:45.210780    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59ff97898d-4tlzq"
Oct 30 00:39:45.211319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:45.210819    6011 scope.go:122] "RemoveContainer" containerID="3b999128fdbb7f9fe304379e317093d6be3fcbab8c5ba5df4074a529b290498b"
Oct 30 00:39:45.260964 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:45.260924    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59ff97898d-4tlzq"]
Oct 30 00:39:45.271106 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:45.271076    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59ff97898d-4tlzq"]
Oct 30 00:39:46.018821 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:46.018785    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="24c36423-7a10-4a80-a3c8-2e3835c3d12a" path="/var/lib/kubelet/pods/24c36423-7a10-4a80-a3c8-2e3835c3d12a/volumes"
Oct 30 00:39:49.285001 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:49.284968    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pgl9q"
Oct 30 00:39:50.223109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.223041    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-64684dbfbb-flsst" podUID="636aa0c9-51c4-43d1-ad08-78922681e9d7" containerName="console" containerID="cri-o://c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59" gracePeriod=15
Oct 30 00:39:50.477589 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.477526    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:50.495394 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495366    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\" (UniqueName: \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\") pod \"636aa0c9-51c4-43d1-ad08-78922681e9d7\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") "
Oct 30 00:39:50.495554 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495415    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\") pod \"636aa0c9-51c4-43d1-ad08-78922681e9d7\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") "
Oct 30 00:39:50.495554 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495447    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\") pod \"636aa0c9-51c4-43d1-ad08-78922681e9d7\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") "
Oct 30 00:39:50.495554 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495475    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\") pod \"636aa0c9-51c4-43d1-ad08-78922681e9d7\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") "
Oct 30 00:39:50.495554 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495519    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\") pod \"636aa0c9-51c4-43d1-ad08-78922681e9d7\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") "
Oct 30 00:39:50.495762 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495600    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\" (UniqueName: \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\") pod \"636aa0c9-51c4-43d1-ad08-78922681e9d7\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") "
Oct 30 00:39:50.495762 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495655    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\") pod \"636aa0c9-51c4-43d1-ad08-78922681e9d7\" (UID: \"636aa0c9-51c4-43d1-ad08-78922681e9d7\") "
Oct 30 00:39:50.495762 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495669    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp" pod "636aa0c9-51c4-43d1-ad08-78922681e9d7" (UID: "636aa0c9-51c4-43d1-ad08-78922681e9d7"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:39:50.495901 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495835    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert" pod "636aa0c9-51c4-43d1-ad08-78922681e9d7" (UID: "636aa0c9-51c4-43d1-ad08-78922681e9d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:39:50.495968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495897    6011 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/636aa0c9-51c4-43d1-ad08-78922681e9d7-tmp\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:50.495968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.495894    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config" pod "636aa0c9-51c4-43d1-ad08-78922681e9d7" (UID: "636aa0c9-51c4-43d1-ad08-78922681e9d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:39:50.496191 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.496145    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca" pod "636aa0c9-51c4-43d1-ad08-78922681e9d7" (UID: "636aa0c9-51c4-43d1-ad08-78922681e9d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:39:50.497980 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.497942    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config" pod "636aa0c9-51c4-43d1-ad08-78922681e9d7" (UID: "636aa0c9-51c4-43d1-ad08-78922681e9d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:39:50.498099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.497974    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj" pod "636aa0c9-51c4-43d1-ad08-78922681e9d7" (UID: "636aa0c9-51c4-43d1-ad08-78922681e9d7"). InnerVolumeSpecName "kube-api-access-qwlfj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:39:50.498099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.498003    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert" pod "636aa0c9-51c4-43d1-ad08-78922681e9d7" (UID: "636aa0c9-51c4-43d1-ad08-78922681e9d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:39:50.596370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.596327    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwlfj\" (UniqueName: \"kubernetes.io/projected/636aa0c9-51c4-43d1-ad08-78922681e9d7-kube-api-access-qwlfj\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:50.596370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.596365    6011 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:50.596370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.596375    6011 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-oauth-serving-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:50.596610 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.596384    6011 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:50.596610 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.596393    6011 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-serving-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:50.596610 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:50.596401    6011 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/636aa0c9-51c4-43d1-ad08-78922681e9d7-console-oauth-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:39:51.236564 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.236527    6011 generic.go:356] "Generic (PLEG): container finished" podID="636aa0c9-51c4-43d1-ad08-78922681e9d7" containerID="c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59" exitCode=0
Oct 30 00:39:51.236564 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.236570    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64684dbfbb-flsst"
Oct 30 00:39:51.236837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.236606    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64684dbfbb-flsst" event={"ID":"636aa0c9-51c4-43d1-ad08-78922681e9d7","Type":"ContainerDied","Data":"c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59"}
Oct 30 00:39:51.236837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.236658    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64684dbfbb-flsst" event={"ID":"636aa0c9-51c4-43d1-ad08-78922681e9d7","Type":"ContainerDied","Data":"207928c3644e47e791f87f307ef73f130c86554b40c72be2a378b132a54dc3bf"}
Oct 30 00:39:51.236837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.236685    6011 scope.go:122] "RemoveContainer" containerID="c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59"
Oct 30 00:39:51.245895 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.245879    6011 scope.go:122] "RemoveContainer" containerID="c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59"
Oct 30 00:39:51.246146 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:51.246127    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59\": container with ID starting with c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59 not found: ID does not exist" containerID="c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59"
Oct 30 00:39:51.246218 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.246155    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59"} err="failed to get container status \"c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59\": rpc error: code = NotFound desc = could not find container \"c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59\": container with ID starting with c175836488dbe56432ae86c568d7ac2df10f64b6d0d3cc1a1341fa393f097f59 not found: ID does not exist"
Oct 30 00:39:51.285137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.285104    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64684dbfbb-flsst"]
Oct 30 00:39:51.297114 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:51.297087    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64684dbfbb-flsst"]
Oct 30 00:39:52.018465 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:52.018430    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="636aa0c9-51c4-43d1-ad08-78922681e9d7" path="/var/lib/kubelet/pods/636aa0c9-51c4-43d1-ad08-78922681e9d7/volumes"
Oct 30 00:39:55.754607 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:55.754577    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:55.758384 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:39:55.758360    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5dc88c6bb9-9zp98"
Oct 30 00:39:57.907878 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:39:57.907835    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:40:14.694539 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:14.694503    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:40:14.695027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:14.694778    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="alertmanager" containerID="cri-o://1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e" gracePeriod=120
Oct 30 00:40:14.695027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:14.694848    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-metric" containerID="cri-o://32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062" gracePeriod=120
Oct 30 00:40:14.695027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:14.694867    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-web" containerID="cri-o://798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe" gracePeriod=120
Oct 30 00:40:14.695027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:14.694936    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy" containerID="cri-o://be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8" gracePeriod=120
Oct 30 00:40:14.695027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:14.694949    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="prom-label-proxy" containerID="cri-o://1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8" gracePeriod=120
Oct 30 00:40:14.695309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:14.694885    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="config-reloader" containerID="cri-o://4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca" gracePeriod=120
Oct 30 00:40:15.330109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330080    6011 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8" exitCode=0
Oct 30 00:40:15.330109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330104    6011 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062" exitCode=0
Oct 30 00:40:15.330109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330111    6011 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8" exitCode=0
Oct 30 00:40:15.330109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330118    6011 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca" exitCode=0
Oct 30 00:40:15.330392 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330125    6011 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e" exitCode=0
Oct 30 00:40:15.330392 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330162    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8"}
Oct 30 00:40:15.330392 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330257    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062"}
Oct 30 00:40:15.330392 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330284    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8"}
Oct 30 00:40:15.330392 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330311    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca"}
Oct 30 00:40:15.330392 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.330336    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e"}
Oct 30 00:40:15.927370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:15.927345    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.012713 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.012394    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.012713 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.012480    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.013587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.012935    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.013587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.013003    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.013587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.013035    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.013587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.013100    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.013587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.013106    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:40:16.013587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.013134    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.014568 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.014501    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.015601    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.015661    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.015702    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.015748    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.015780    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.015814    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\") pod \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\" (UID: \"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16\") "
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.016062    6011 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.016358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.016082    6011 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-alertmanager-main-db\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.016800 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.016490    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:40:16.018458 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.018429    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "kube-api-access-fn4jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:40:16.019250 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.019203    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:40:16.020431 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.020410    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:40:16.020719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.020686    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:40:16.020896 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.020856    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:40:16.021133 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.021101    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:40:16.021230 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.021167    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:40:16.025452 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.025427    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:40:16.027661 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.027634    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:40:16.034898 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.034868    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config" pod "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" (UID: "1a9bfb9c-bc15-48d5-98a3-5bd1761fde16"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:40:16.116879 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116842    6011 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-out\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.116879 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116871    6011 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.116879 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116881    6011 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.116879 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116890    6011 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-web-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.117147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116899    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fn4jb\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-kube-api-access-fn4jb\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.117147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116908    6011 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-main-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.117147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116919    6011 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-tls-assets\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.117147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116928    6011 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.117147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116936    6011 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-cluster-tls-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.117147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116945    6011 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.117147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.116953    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16-metrics-client-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:40:16.337164 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.337126    6011 generic.go:356] "Generic (PLEG): container finished" podID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerID="798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe" exitCode=0
Oct 30 00:40:16.337370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.337220    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe"}
Oct 30 00:40:16.337370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.337292    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"1a9bfb9c-bc15-48d5-98a3-5bd1761fde16","Type":"ContainerDied","Data":"f45aa391f5719f6f864ace1ec9acd9b6f486d32cc18c563440b86a6bccb6f117"}
Oct 30 00:40:16.337370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.337228    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.337370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.337323    6011 scope.go:122] "RemoveContainer" containerID="1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8"
Oct 30 00:40:16.345837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.345823    6011 scope.go:122] "RemoveContainer" containerID="32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062"
Oct 30 00:40:16.353011 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.352991    6011 scope.go:122] "RemoveContainer" containerID="be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8"
Oct 30 00:40:16.360066 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.360045    6011 scope.go:122] "RemoveContainer" containerID="798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe"
Oct 30 00:40:16.366658 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.366629    6011 scope.go:122] "RemoveContainer" containerID="4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca"
Oct 30 00:40:16.373315 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.373296    6011 scope.go:122] "RemoveContainer" containerID="1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e"
Oct 30 00:40:16.380100 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.380069    6011 scope.go:122] "RemoveContainer" containerID="4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f"
Oct 30 00:40:16.387041 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.387022    6011 scope.go:122] "RemoveContainer" containerID="1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8"
Oct 30 00:40:16.387464 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:16.387437    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8\": container with ID starting with 1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8 not found: ID does not exist" containerID="1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8"
Oct 30 00:40:16.387541 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.387484    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8"} err="failed to get container status \"1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8\": rpc error: code = NotFound desc = could not find container \"1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8\": container with ID starting with 1760c2c69d39af0bf714eba664047cf58ad6c74f4c085f7761377b87701120a8 not found: ID does not exist"
Oct 30 00:40:16.387541 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.387512    6011 scope.go:122] "RemoveContainer" containerID="32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062"
Oct 30 00:40:16.387792 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:16.387775    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062\": container with ID starting with 32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062 not found: ID does not exist" containerID="32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062"
Oct 30 00:40:16.387855 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.387801    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062"} err="failed to get container status \"32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062\": rpc error: code = NotFound desc = could not find container \"32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062\": container with ID starting with 32c4901a9cb78b9f209f4c183bf6dc767b53e30f8a51efded62817aa5d076062 not found: ID does not exist"
Oct 30 00:40:16.387855 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.387820    6011 scope.go:122] "RemoveContainer" containerID="be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8"
Oct 30 00:40:16.388105 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:16.388089    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8\": container with ID starting with be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8 not found: ID does not exist" containerID="be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8"
Oct 30 00:40:16.388165 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.388119    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8"} err="failed to get container status \"be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8\": rpc error: code = NotFound desc = could not find container \"be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8\": container with ID starting with be904b3b6a4f923a4ef80203100a3ff4ec54eec45670e53f4992d8c3cbedf9f8 not found: ID does not exist"
Oct 30 00:40:16.388165 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.388133    6011 scope.go:122] "RemoveContainer" containerID="798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe"
Oct 30 00:40:16.388465 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:16.388449    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe\": container with ID starting with 798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe not found: ID does not exist" containerID="798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe"
Oct 30 00:40:16.388520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.388470    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe"} err="failed to get container status \"798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe\": rpc error: code = NotFound desc = could not find container \"798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe\": container with ID starting with 798b95e6c4f32a8c36aa567a8ea3e0fee04977abc42dcceb4fec1f170f8124fe not found: ID does not exist"
Oct 30 00:40:16.388520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.388485    6011 scope.go:122] "RemoveContainer" containerID="4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca"
Oct 30 00:40:16.388850 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:16.388824    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca\": container with ID starting with 4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca not found: ID does not exist" containerID="4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca"
Oct 30 00:40:16.388920 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.388856    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca"} err="failed to get container status \"4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca\": rpc error: code = NotFound desc = could not find container \"4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca\": container with ID starting with 4989250c00288584432689d8db67c5951a7213a75da0f52ed13a0d9d285d28ca not found: ID does not exist"
Oct 30 00:40:16.388920 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.388882    6011 scope.go:122] "RemoveContainer" containerID="1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e"
Oct 30 00:40:16.389161 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:16.389142    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e\": container with ID starting with 1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e not found: ID does not exist" containerID="1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e"
Oct 30 00:40:16.389229 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.389165    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e"} err="failed to get container status \"1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e\": rpc error: code = NotFound desc = could not find container \"1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e\": container with ID starting with 1d3e490759ebec0966ec950137c619e36c82e8233fbb8dd5180cd334fc45888e not found: ID does not exist"
Oct 30 00:40:16.389229 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.389196    6011 scope.go:122] "RemoveContainer" containerID="4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f"
Oct 30 00:40:16.389467 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:16.389445    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f\": container with ID starting with 4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f not found: ID does not exist" containerID="4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f"
Oct 30 00:40:16.389537 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.389484    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f"} err="failed to get container status \"4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f\": rpc error: code = NotFound desc = could not find container \"4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f\": container with ID starting with 4cc04ce5e37382ed160864cded8d8fc543f702c21baf33df038641c1a684c56f not found: ID does not exist"
Oct 30 00:40:16.390331 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.390316    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:40:16.402454 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.402427    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:40:16.427524 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427503    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:40:16.427907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427885    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-web"
Oct 30 00:40:16.427907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427908    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-web"
Oct 30 00:40:16.427907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427920    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="init-config-reloader"
Oct 30 00:40:16.427907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427928    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="init-config-reloader"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427945    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="alertmanager"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427953    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="alertmanager"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427966    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="config-reloader"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427973    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="config-reloader"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.427995    6011 cpu_manager.go:414] "containerMap: removing container" podUID="636aa0c9-51c4-43d1-ad08-78922681e9d7" containerName="console"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428001    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="636aa0c9-51c4-43d1-ad08-78922681e9d7" containerName="console"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428013    6011 cpu_manager.go:414] "containerMap: removing container" podUID="24c36423-7a10-4a80-a3c8-2e3835c3d12a" containerName="console"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428019    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="24c36423-7a10-4a80-a3c8-2e3835c3d12a" containerName="console"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428030    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="prom-label-proxy"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428036    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="prom-label-proxy"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428044    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428050    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428071    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-metric"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428078    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-metric"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428144    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="config-reloader"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428155    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="636aa0c9-51c4-43d1-ad08-78922681e9d7" containerName="console"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428163    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="24c36423-7a10-4a80-a3c8-2e3835c3d12a" containerName="console"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428188    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="prom-label-proxy"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428197    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428207    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-metric"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428214    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="kube-rbac-proxy-web"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428227    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="init-config-reloader"
Oct 30 00:40:16.428307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.428236    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" containerName="alertmanager"
Oct 30 00:40:16.432188 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.432162    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.447486 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447457    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 00:40:16.447486 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447467    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 00:40:16.447660 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447558    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 00:40:16.447720 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447664    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 00:40:16.447720 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447691    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 00:40:16.447720 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447690    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 00:40:16.447815 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447708    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-lpt8z\""
Oct 30 00:40:16.447815 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447774    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 00:40:16.447985 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.447972    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 00:40:16.459982 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.459951    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 00:40:16.460574 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.460553    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:40:16.520684 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.520640    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/34a9b08b-9aed-4952-abdd-41713e001f35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.520684 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.520694    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.520892 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.520722    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-config-volume\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.520892 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.520859    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwthz\" (UniqueName: \"kubernetes.io/projected/34a9b08b-9aed-4952-abdd-41713e001f35-kube-api-access-dwthz\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.520955 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.520890    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-web-config\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.520991 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.520954    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.520991 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.520986    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.521057 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.521004    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.521057 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.521033    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.521117 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.521065    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34a9b08b-9aed-4952-abdd-41713e001f35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.521152 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.521136    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/34a9b08b-9aed-4952-abdd-41713e001f35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.521215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.521158    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34a9b08b-9aed-4952-abdd-41713e001f35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.521215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.521189    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/34a9b08b-9aed-4952-abdd-41713e001f35-config-out\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.621996 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.621907    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/34a9b08b-9aed-4952-abdd-41713e001f35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.621996 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.621951    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34a9b08b-9aed-4952-abdd-41713e001f35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.621996 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.621968    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/34a9b08b-9aed-4952-abdd-41713e001f35-config-out\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.621996 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.621998    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/34a9b08b-9aed-4952-abdd-41713e001f35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622025    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622043    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-config-volume\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622199    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwthz\" (UniqueName: \"kubernetes.io/projected/34a9b08b-9aed-4952-abdd-41713e001f35-kube-api-access-dwthz\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622237    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-web-config\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622271    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622309    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622337    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622664 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622363    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622664 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622395    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34a9b08b-9aed-4952-abdd-41713e001f35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622664 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622555    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/34a9b08b-9aed-4952-abdd-41713e001f35-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.622664 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622553    6011 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 00:40:16.622848 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.622813    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/34a9b08b-9aed-4952-abdd-41713e001f35-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.623601 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.623553    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/34a9b08b-9aed-4952-abdd-41713e001f35-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.625945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.625920    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.626065 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.626014    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-config-volume\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.626319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.626269    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.626319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.626304    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.626498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.626374    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.626498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.626431    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/34a9b08b-9aed-4952-abdd-41713e001f35-tls-assets\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.626498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.626454    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.626498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.626477    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/34a9b08b-9aed-4952-abdd-41713e001f35-web-config\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.627288 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.627271    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/34a9b08b-9aed-4952-abdd-41713e001f35-config-out\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.647195 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.647158    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwthz\" (UniqueName: \"kubernetes.io/projected/34a9b08b-9aed-4952-abdd-41713e001f35-kube-api-access-dwthz\") pod \"alertmanager-main-0\" (UID: \"34a9b08b-9aed-4952-abdd-41713e001f35\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:16.748987 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:16.748957    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 00:40:17.078014 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:17.077637    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 00:40:17.080775 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:40:17.080735    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34a9b08b_9aed_4952_abdd_41713e001f35.slice/crio-ed4730efba4398f9e5839429c19d9896a192d5efb3beb810cdc02ed99a3f23ab WatchSource:0}: Error finding container ed4730efba4398f9e5839429c19d9896a192d5efb3beb810cdc02ed99a3f23ab: Status 404 returned error can't find the container with id ed4730efba4398f9e5839429c19d9896a192d5efb3beb810cdc02ed99a3f23ab
Oct 30 00:40:17.343760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:17.343674    6011 generic.go:356] "Generic (PLEG): container finished" podID="34a9b08b-9aed-4952-abdd-41713e001f35" containerID="3610717dbafa1fb610b84422d0bbbc27e4de2ace6eb989a957b15f93fe830c51" exitCode=0
Oct 30 00:40:17.343760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:17.343713    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerDied","Data":"3610717dbafa1fb610b84422d0bbbc27e4de2ace6eb989a957b15f93fe830c51"}
Oct 30 00:40:17.343915 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:17.343792    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerStarted","Data":"ed4730efba4398f9e5839429c19d9896a192d5efb3beb810cdc02ed99a3f23ab"}
Oct 30 00:40:18.017952 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.017918    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a9bfb9c-bc15-48d5-98a3-5bd1761fde16" path="/var/lib/kubelet/pods/1a9bfb9c-bc15-48d5-98a3-5bd1761fde16/volumes"
Oct 30 00:40:18.351602 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.351565    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerStarted","Data":"47ae9b834458b63497d750ea9f4b29af9e77d573e0c0ba1e7befd856bc020035"}
Oct 30 00:40:18.352030 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.351655    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerStarted","Data":"66e20da5677afd731cc9355807a199a0bab847fb61f5ef1435585d459e6e7fcb"}
Oct 30 00:40:18.352030 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.351681    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerStarted","Data":"873990a8cb711e423f02e164c2e2743202e8e3457836947a53a4b975d950784c"}
Oct 30 00:40:18.352030 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.351705    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerStarted","Data":"1d77dea6d0bab100083a31bad7b2543d271182017c7b37119bde653849f1b7c0"}
Oct 30 00:40:18.352030 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.351731    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerStarted","Data":"226acabcc9ea36dfd29df9b69d4059258b67527f90e75ad76cf11b831d953d88"}
Oct 30 00:40:18.352030 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.351757    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"34a9b08b-9aed-4952-abdd-41713e001f35","Type":"ContainerStarted","Data":"ebd4b994dd5179f8153ac71aa6c9bce68a0b769fcfa242c5c5b2f35900473651"}
Oct 30 00:40:18.413736 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:18.413674    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.4007434180000002 podStartE2EDuration="2.413647238s" totalImagesPullingTime="0s" totalInitContainerRuntime="12.90382ms" isStatefulPod=false podCreationTimestamp="2026-10-30 00:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:40:18.401244225 +0000 UTC m=+60.503998199" watchObservedRunningTime="2026-10-30 00:40:18.413647238 +0000 UTC m=+60.516401210"
Oct 30 00:40:27.908681 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:27.908644    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:40:29.840760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.840726    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-d6469dd46-qmpmz"]
Oct 30 00:40:29.842840 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.842818    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.873822 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.873790    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d6469dd46-qmpmz"]
Oct 30 00:40:29.940228 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940191    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-trusted-ca-bundle\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.940396 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940249    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7228a710-717e-476f-beac-43807677b0c1-console-serving-cert\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.940396 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940283    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7228a710-717e-476f-beac-43807677b0c1-console-oauth-config\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.940396 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940343    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-console-config\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.940396 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940381    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-service-ca\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.940531 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940448    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-oauth-serving-cert\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.940531 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940465    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs4nb\" (UniqueName: \"kubernetes.io/projected/7228a710-717e-476f-beac-43807677b0c1-kube-api-access-fs4nb\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:29.940531 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:29.940495    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7228a710-717e-476f-beac-43807677b0c1-tmp\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.040871 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.040844    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-console-config\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041070 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.040887    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-service-ca\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041070 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.040909    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-oauth-serving-cert\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041070 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041017    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs4nb\" (UniqueName: \"kubernetes.io/projected/7228a710-717e-476f-beac-43807677b0c1-kube-api-access-fs4nb\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041070 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041066    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7228a710-717e-476f-beac-43807677b0c1-tmp\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041123    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-trusted-ca-bundle\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041164    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7228a710-717e-476f-beac-43807677b0c1-console-serving-cert\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041205    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7228a710-717e-476f-beac-43807677b0c1-console-oauth-config\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041588 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041560    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7228a710-717e-476f-beac-43807677b0c1-tmp\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041658 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041625    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-service-ca\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041758 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041740    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-oauth-serving-cert\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.041838 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041814    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-console-config\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.042011 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.041987    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7228a710-717e-476f-beac-43807677b0c1-trusted-ca-bundle\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.043794 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.043771    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7228a710-717e-476f-beac-43807677b0c1-console-serving-cert\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.043982 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.043799    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7228a710-717e-476f-beac-43807677b0c1-console-oauth-config\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.064527 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.064497    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs4nb\" (UniqueName: \"kubernetes.io/projected/7228a710-717e-476f-beac-43807677b0c1-kube-api-access-fs4nb\") pod \"console-d6469dd46-qmpmz\" (UID: \"7228a710-717e-476f-beac-43807677b0c1\") " pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.153097 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.153007    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:30.454622 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:30.454534    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d6469dd46-qmpmz"]
Oct 30 00:40:30.457443 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:40:30.457408    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7228a710_717e_476f_beac_43807677b0c1.slice/crio-21e8ac9e5619cd4143ad483e495565d1364b431e7980823de7ef337a79feb938 WatchSource:0}: Error finding container 21e8ac9e5619cd4143ad483e495565d1364b431e7980823de7ef337a79feb938: Status 404 returned error can't find the container with id 21e8ac9e5619cd4143ad483e495565d1364b431e7980823de7ef337a79feb938
Oct 30 00:40:31.402150 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:31.402111    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d6469dd46-qmpmz" event={"ID":"7228a710-717e-476f-beac-43807677b0c1","Type":"ContainerStarted","Data":"74da8d00c91f0080d63afb6fc59c7cff04591228e424d7aee9e3cb3de3fca11a"}
Oct 30 00:40:31.402576 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:31.402221    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d6469dd46-qmpmz" event={"ID":"7228a710-717e-476f-beac-43807677b0c1","Type":"ContainerStarted","Data":"21e8ac9e5619cd4143ad483e495565d1364b431e7980823de7ef337a79feb938"}
Oct 30 00:40:31.455425 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:31.455364    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-d6469dd46-qmpmz" podStartSLOduration=2.455349423 podStartE2EDuration="2.455349423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:40:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:40:31.454314225 +0000 UTC m=+73.557068197" watchObservedRunningTime="2026-10-30 00:40:31.455349423 +0000 UTC m=+73.558103385"
Oct 30 00:40:40.153663 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:40.153630    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:40.154063 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:40.153925    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:40.158342 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:40.158322    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:40.437630 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:40.437538    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d6469dd46-qmpmz"
Oct 30 00:40:40.505355 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:40:40.505320    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78cf5d5fdb-fxvv5"]
Oct 30 00:40:57.908872 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:40:57.908828    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:41:05.526056 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.525999    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-78cf5d5fdb-fxvv5" podUID="0b672b37-e2e1-42f5-8adc-001a7ec77185" containerName="console" containerID="cri-o://ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7" gracePeriod=15
Oct 30 00:41:05.763966 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.763943    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:41:05.867606 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867562    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.867606 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867613    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.867873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867658    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.867873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867689    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\" (UniqueName: \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.867873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867732    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.867873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867766    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.867873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867791    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.867873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.867861    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\") pod \"0b672b37-e2e1-42f5-8adc-001a7ec77185\" (UID: \"0b672b37-e2e1-42f5-8adc-001a7ec77185\") "
Oct 30 00:41:05.868152 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.868102    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:41:05.868290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.868147    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:41:05.868355 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.868300    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:41:05.868415 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.868362    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:41:05.868415 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.868403    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:41:05.870163 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.870139    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:41:05.870324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.870243    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "kube-api-access-8229s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:41:05.870324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.870262    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert" pod "0b672b37-e2e1-42f5-8adc-001a7ec77185" (UID: "0b672b37-e2e1-42f5-8adc-001a7ec77185"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:41:05.969537 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969497    6011 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b672b37-e2e1-42f5-8adc-001a7ec77185-tmp\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:05.969537 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969528    6011 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-trusted-ca-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:05.969537 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969542    6011 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:05.969841 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969554    6011 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-oauth-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:05.969841 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969566    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8229s\" (UniqueName: \"kubernetes.io/projected/0b672b37-e2e1-42f5-8adc-001a7ec77185-kube-api-access-8229s\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:05.969841 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969578    6011 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-serving-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:05.969841 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969589    6011 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-console-config\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:05.969841 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:05.969600    6011 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b672b37-e2e1-42f5-8adc-001a7ec77185-oauth-serving-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:41:06.530490 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.530451    6011 generic.go:356] "Generic (PLEG): container finished" podID="0b672b37-e2e1-42f5-8adc-001a7ec77185" containerID="ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7" exitCode=0
Oct 30 00:41:06.530873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.530502    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78cf5d5fdb-fxvv5"
Oct 30 00:41:06.530873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.530534    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf5d5fdb-fxvv5" event={"ID":"0b672b37-e2e1-42f5-8adc-001a7ec77185","Type":"ContainerDied","Data":"ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7"}
Oct 30 00:41:06.530873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.530581    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78cf5d5fdb-fxvv5" event={"ID":"0b672b37-e2e1-42f5-8adc-001a7ec77185","Type":"ContainerDied","Data":"db67bdbb04cc28a261d52554b33192b8da20975fe2ef2cf082ea275dcaca59a0"}
Oct 30 00:41:06.530873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.530605    6011 scope.go:122] "RemoveContainer" containerID="ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7"
Oct 30 00:41:06.538741 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.538721    6011 scope.go:122] "RemoveContainer" containerID="ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7"
Oct 30 00:41:06.539045 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:41:06.539027    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7\": container with ID starting with ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7 not found: ID does not exist" containerID="ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7"
Oct 30 00:41:06.539099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.539055    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7"} err="failed to get container status \"ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7\": rpc error: code = NotFound desc = could not find container \"ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7\": container with ID starting with ea3215ea8ba7e283faa1331c109b19044eb03a4c97f722717c02c81bc850f2c7 not found: ID does not exist"
Oct 30 00:41:06.579923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.579892    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78cf5d5fdb-fxvv5"]
Oct 30 00:41:06.591976 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:06.591952    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78cf5d5fdb-fxvv5"]
Oct 30 00:41:08.018805 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:41:08.018768    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0b672b37-e2e1-42f5-8adc-001a7ec77185" path="/var/lib/kubelet/pods/0b672b37-e2e1-42f5-8adc-001a7ec77185/volumes"
Oct 30 00:41:27.908221 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:41:27.908161    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:41:57.909269 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:41:57.909228    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:42:27.908612 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:42:27.908531    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:42:38.735898 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.735858    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"]
Oct 30 00:42:38.736444 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.736264    6011 cpu_manager.go:414] "containerMap: removing container" podUID="0b672b37-e2e1-42f5-8adc-001a7ec77185" containerName="console"
Oct 30 00:42:38.736444 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.736278    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0b672b37-e2e1-42f5-8adc-001a7ec77185" containerName="console"
Oct 30 00:42:38.736444 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.736397    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="0b672b37-e2e1-42f5-8adc-001a7ec77185" containerName="console"
Oct 30 00:42:38.739640 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.739624    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.755448 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.755410    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 00:42:38.767337 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.767306    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 00:42:38.767497 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.767469    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-qg8zf\""
Oct 30 00:42:38.767998 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.767979    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"]
Oct 30 00:42:38.825228 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.825161    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.825410 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.825355    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr8kb\" (UniqueName: \"kubernetes.io/projected/9d1ed35d-5b56-4858-9f87-11a10518f41b-kube-api-access-vr8kb\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.825410 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.825402    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.926873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.926841    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.927044 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.926889    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.927044 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.926948    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr8kb\" (UniqueName: \"kubernetes.io/projected/9d1ed35d-5b56-4858-9f87-11a10518f41b-kube-api-access-vr8kb\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.927251 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.927230    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.927298 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.927272    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:38.946635 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:38.946604    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr8kb\" (UniqueName: \"kubernetes.io/projected/9d1ed35d-5b56-4858-9f87-11a10518f41b-kube-api-access-vr8kb\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:39.054580 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:39.054552    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:39.346096 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:39.346019    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"]
Oct 30 00:42:39.349426 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:42:39.349394    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d1ed35d_5b56_4858_9f87_11a10518f41b.slice/crio-8aa131a3672186e4ce342a4ba7c75b563c92eb1a7a86de79197993180e96528b WatchSource:0}: Error finding container 8aa131a3672186e4ce342a4ba7c75b563c92eb1a7a86de79197993180e96528b: Status 404 returned error can't find the container with id 8aa131a3672186e4ce342a4ba7c75b563c92eb1a7a86de79197993180e96528b
Oct 30 00:42:39.351526 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:39.351507    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:42:39.868818 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:39.868751    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7" event={"ID":"9d1ed35d-5b56-4858-9f87-11a10518f41b","Type":"ContainerStarted","Data":"8aa131a3672186e4ce342a4ba7c75b563c92eb1a7a86de79197993180e96528b"}
Oct 30 00:42:44.891018 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:44.890984    6011 generic.go:356] "Generic (PLEG): container finished" podID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerID="589e24cfd65f97494bbd6864347e84d60b6373fe70c5992482013cd8e83adb2d" exitCode=0
Oct 30 00:42:44.891443 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:44.891072    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7" event={"ID":"9d1ed35d-5b56-4858-9f87-11a10518f41b","Type":"ContainerDied","Data":"589e24cfd65f97494bbd6864347e84d60b6373fe70c5992482013cd8e83adb2d"}
Oct 30 00:42:47.905497 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:47.905463    6011 generic.go:356] "Generic (PLEG): container finished" podID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerID="e2f45f2fc69ced623621d74d26fad9930040bfe2bdc10df710adc5dd94ad974c" exitCode=0
Oct 30 00:42:47.905907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:47.905536    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7" event={"ID":"9d1ed35d-5b56-4858-9f87-11a10518f41b","Type":"ContainerDied","Data":"e2f45f2fc69ced623621d74d26fad9930040bfe2bdc10df710adc5dd94ad974c"}
Oct 30 00:42:53.935788 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:53.935756    6011 generic.go:356] "Generic (PLEG): container finished" podID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerID="0abf5ae24134079efab337885b699723d9a8a9725e17f08e9e7dd8767705edfb" exitCode=0
Oct 30 00:42:53.936288 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:53.935840    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7" event={"ID":"9d1ed35d-5b56-4858-9f87-11a10518f41b","Type":"ContainerDied","Data":"0abf5ae24134079efab337885b699723d9a8a9725e17f08e9e7dd8767705edfb"}
Oct 30 00:42:55.072743 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.072709    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:55.179036 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.178999    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d1ed35d-5b56-4858-9f87-11a10518f41b-kube-api-access-vr8kb\" (UniqueName: \"kubernetes.io/projected/9d1ed35d-5b56-4858-9f87-11a10518f41b-kube-api-access-vr8kb\") pod \"9d1ed35d-5b56-4858-9f87-11a10518f41b\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") "
Oct 30 00:42:55.179261 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.179081    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-util\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-util\") pod \"9d1ed35d-5b56-4858-9f87-11a10518f41b\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") "
Oct 30 00:42:55.179261 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.179105    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-bundle\") pod \"9d1ed35d-5b56-4858-9f87-11a10518f41b\" (UID: \"9d1ed35d-5b56-4858-9f87-11a10518f41b\") "
Oct 30 00:42:55.179837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.179803    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-bundle" pod "9d1ed35d-5b56-4858-9f87-11a10518f41b" (UID: "9d1ed35d-5b56-4858-9f87-11a10518f41b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:42:55.181717 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.181696    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d1ed35d-5b56-4858-9f87-11a10518f41b-kube-api-access-vr8kb" pod "9d1ed35d-5b56-4858-9f87-11a10518f41b" (UID: "9d1ed35d-5b56-4858-9f87-11a10518f41b"). InnerVolumeSpecName "kube-api-access-vr8kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:42:55.183993 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.183974    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-util" pod "9d1ed35d-5b56-4858-9f87-11a10518f41b" (UID: "9d1ed35d-5b56-4858-9f87-11a10518f41b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:42:55.279790 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.279701    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vr8kb\" (UniqueName: \"kubernetes.io/projected/9d1ed35d-5b56-4858-9f87-11a10518f41b-kube-api-access-vr8kb\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:42:55.279790 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.279732    6011 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-util\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:42:55.279790 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.279741    6011 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d1ed35d-5b56-4858-9f87-11a10518f41b-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:42:55.945061 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.945021    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7" event={"ID":"9d1ed35d-5b56-4858-9f87-11a10518f41b","Type":"ContainerDied","Data":"8aa131a3672186e4ce342a4ba7c75b563c92eb1a7a86de79197993180e96528b"}
Oct 30 00:42:55.945061 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.945069    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8aa131a3672186e4ce342a4ba7c75b563c92eb1a7a86de79197993180e96528b"
Oct 30 00:42:55.945354 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:42:55.945083    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cbw8x7"
Oct 30 00:42:57.908060 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:42:57.908017    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:43:01.917749 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.917707    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"]
Oct 30 00:43:01.918161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.918082    6011 cpu_manager.go:414] "containerMap: removing container" podUID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerName="extract"
Oct 30 00:43:01.918161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.918095    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerName="extract"
Oct 30 00:43:01.918161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.918108    6011 cpu_manager.go:414] "containerMap: removing container" podUID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerName="pull"
Oct 30 00:43:01.918161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.918113    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerName="pull"
Oct 30 00:43:01.918161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.918135    6011 cpu_manager.go:414] "containerMap: removing container" podUID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerName="util"
Oct 30 00:43:01.918161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.918140    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerName="util"
Oct 30 00:43:01.918417 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.918214    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d1ed35d-5b56-4858-9f87-11a10518f41b" containerName="extract"
Oct 30 00:43:01.959100 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.959064    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"]
Oct 30 00:43:01.959292 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.959187    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:01.975290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.975268    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Oct 30 00:43:01.975290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.975268    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:43:01.975290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:01.975268    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-cm4l8\""
Oct 30 00:43:02.037990 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.037960    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-ffg6b\" (UID: \"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:02.037990 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.037997    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25zhk\" (UniqueName: \"kubernetes.io/projected/73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8-kube-api-access-25zhk\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-ffg6b\" (UID: \"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:02.139235 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.139198    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-ffg6b\" (UID: \"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:02.139437 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.139245    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25zhk\" (UniqueName: \"kubernetes.io/projected/73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8-kube-api-access-25zhk\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-ffg6b\" (UID: \"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:02.139600 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.139573    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8-tmp\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-ffg6b\" (UID: \"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:02.158799 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.158763    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25zhk\" (UniqueName: \"kubernetes.io/projected/73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8-kube-api-access-25zhk\") pod \"cert-manager-operator-controller-manager-89fdbf7dc-ffg6b\" (UID: \"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:02.283298 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.283267    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"
Oct 30 00:43:02.626324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.626248    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b"]
Oct 30 00:43:02.629874 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:02.629843    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c4ca6b_3f44_40b7_8cbf_1d2476a8ebd8.slice/crio-65cd88d18902ab7fed0579e3229781e8193286a0a2161d370607d5cf9236a3d5 WatchSource:0}: Error finding container 65cd88d18902ab7fed0579e3229781e8193286a0a2161d370607d5cf9236a3d5: Status 404 returned error can't find the container with id 65cd88d18902ab7fed0579e3229781e8193286a0a2161d370607d5cf9236a3d5
Oct 30 00:43:02.973210 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:02.973104    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b" event={"ID":"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8","Type":"ContainerStarted","Data":"65cd88d18902ab7fed0579e3229781e8193286a0a2161d370607d5cf9236a3d5"}
Oct 30 00:43:05.987083 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:05.987050    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b" event={"ID":"73c4ca6b-3f44-40b7-8cbf-1d2476a8ebd8","Type":"ContainerStarted","Data":"96375777ec4d66119e662f5462ad3d94bba7745aca3b9f8b56c09685d9a4c0d3"}
Oct 30 00:43:06.052252 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.052194    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-89fdbf7dc-ffg6b" podStartSLOduration=2.335537802 podStartE2EDuration="5.052168207s" totalImagesPullingTime="2.716630405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:43:06.039668978 +0000 UTC m=+228.142422948" watchObservedRunningTime="2026-10-30 00:43:06.052168207 +0000 UTC m=+228.154922177"
Oct 30 00:43:06.743379 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.743344    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"]
Oct 30 00:43:06.757927 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.757908    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.767840 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.767812    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"]
Oct 30 00:43:06.779321 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.779299    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 00:43:06.779442 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.779351    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 00:43:06.787471 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.787449    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"61b1ff68c7255ee5c5329fe19a5d96293a9578-ce91a85d-dockercfg-9k7kh\""
Oct 30 00:43:06.883400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.883360    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgbst\" (UniqueName: \"kubernetes.io/projected/53232519-8f82-4f9b-98d9-ebcfdab98045-kube-api-access-vgbst\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.883400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.883400    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.883613 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.883494    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.984911 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.984875    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.985086 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.984977    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgbst\" (UniqueName: \"kubernetes.io/projected/53232519-8f82-4f9b-98d9-ebcfdab98045-kube-api-access-vgbst\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.985086 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.985000    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.985371 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.985338    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-bundle\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:06.985447 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:06.985388    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-util\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:07.004981 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:07.004914    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgbst\" (UniqueName: \"kubernetes.io/projected/53232519-8f82-4f9b-98d9-ebcfdab98045-kube-api-access-vgbst\") pod \"61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") " pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:07.108559 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:07.108521    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:07.418525 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:07.418490    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"]
Oct 30 00:43:07.421915 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:07.421887    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53232519_8f82_4f9b_98d9_ebcfdab98045.slice/crio-c6a7e23099d492f1904f2541d176a3eb3211bce99ee7243f1ed18fc24a403d35 WatchSource:0}: Error finding container c6a7e23099d492f1904f2541d176a3eb3211bce99ee7243f1ed18fc24a403d35: Status 404 returned error can't find the container with id c6a7e23099d492f1904f2541d176a3eb3211bce99ee7243f1ed18fc24a403d35
Oct 30 00:43:07.996318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:07.996282    6011 generic.go:356] "Generic (PLEG): container finished" podID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerID="efd9cb6a15ad920003d3c7e14fb7c7d89325edae168aa70105c84ecb400662ce" exitCode=0
Oct 30 00:43:07.996501 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:07.996370    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr" event={"ID":"53232519-8f82-4f9b-98d9-ebcfdab98045","Type":"ContainerDied","Data":"efd9cb6a15ad920003d3c7e14fb7c7d89325edae168aa70105c84ecb400662ce"}
Oct 30 00:43:07.996501 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:07.996417    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr" event={"ID":"53232519-8f82-4f9b-98d9-ebcfdab98045","Type":"ContainerStarted","Data":"c6a7e23099d492f1904f2541d176a3eb3211bce99ee7243f1ed18fc24a403d35"}
Oct 30 00:43:09.145717 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.145676    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"]
Oct 30 00:43:09.148484 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.148459    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.175570 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.175521    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 00:43:09.175787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.175584    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-mztvq\""
Oct 30 00:43:09.175956 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.175935    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 00:43:09.176511 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.176490    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"]
Oct 30 00:43:09.305396 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.305356    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21395c96-946f-4852-86ff-e0518cced3e3-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-gfw77\" (UID: \"21395c96-946f-4852-86ff-e0518cced3e3\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.305590 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.305408    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl9zn\" (UniqueName: \"kubernetes.io/projected/21395c96-946f-4852-86ff-e0518cced3e3-kube-api-access-fl9zn\") pod \"cert-manager-cainjector-5c57bbd4fc-gfw77\" (UID: \"21395c96-946f-4852-86ff-e0518cced3e3\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.406416 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.406331    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21395c96-946f-4852-86ff-e0518cced3e3-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-gfw77\" (UID: \"21395c96-946f-4852-86ff-e0518cced3e3\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.406416 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.406381    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl9zn\" (UniqueName: \"kubernetes.io/projected/21395c96-946f-4852-86ff-e0518cced3e3-kube-api-access-fl9zn\") pod \"cert-manager-cainjector-5c57bbd4fc-gfw77\" (UID: \"21395c96-946f-4852-86ff-e0518cced3e3\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.425474 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.425442    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21395c96-946f-4852-86ff-e0518cced3e3-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-gfw77\" (UID: \"21395c96-946f-4852-86ff-e0518cced3e3\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.425635 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.425476    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl9zn\" (UniqueName: \"kubernetes.io/projected/21395c96-946f-4852-86ff-e0518cced3e3-kube-api-access-fl9zn\") pod \"cert-manager-cainjector-5c57bbd4fc-gfw77\" (UID: \"21395c96-946f-4852-86ff-e0518cced3e3\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.469751 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.469714    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"
Oct 30 00:43:09.776402 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:09.776336    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77"]
Oct 30 00:43:09.779976 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:09.779940    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21395c96_946f_4852_86ff_e0518cced3e3.slice/crio-4b1ba1f8cb16e51a795e99d289e63c9d4eb21ac697127722124183370523eee3 WatchSource:0}: Error finding container 4b1ba1f8cb16e51a795e99d289e63c9d4eb21ac697127722124183370523eee3: Status 404 returned error can't find the container with id 4b1ba1f8cb16e51a795e99d289e63c9d4eb21ac697127722124183370523eee3
Oct 30 00:43:10.006817 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:10.006763    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77" event={"ID":"21395c96-946f-4852-86ff-e0518cced3e3","Type":"ContainerStarted","Data":"4b1ba1f8cb16e51a795e99d289e63c9d4eb21ac697127722124183370523eee3"}
Oct 30 00:43:11.013699 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:11.013660    6011 generic.go:356] "Generic (PLEG): container finished" podID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerID="50572a141ba4be50646579e4d65784b3f437ba5dfc787c38f5c97dbe8e927239" exitCode=0
Oct 30 00:43:11.014143 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:11.013747    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr" event={"ID":"53232519-8f82-4f9b-98d9-ebcfdab98045","Type":"ContainerDied","Data":"50572a141ba4be50646579e4d65784b3f437ba5dfc787c38f5c97dbe8e927239"}
Oct 30 00:43:12.023098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:12.023041    6011 generic.go:356] "Generic (PLEG): container finished" podID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerID="ea3f92fab5ea1697dd2ef9dfe5512585f66742f954d647de0a419a035013bb49" exitCode=0
Oct 30 00:43:12.023098 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:12.023102    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr" event={"ID":"53232519-8f82-4f9b-98d9-ebcfdab98045","Type":"ContainerDied","Data":"ea3f92fab5ea1697dd2ef9dfe5512585f66742f954d647de0a419a035013bb49"}
Oct 30 00:43:13.028530 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.028493    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77" event={"ID":"21395c96-946f-4852-86ff-e0518cced3e3","Type":"ContainerStarted","Data":"ccf49d108f274e3517236577de3ef4559e1abce876e34e15f891a8055fe3d391"}
Oct 30 00:43:13.065644 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.065585    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-gfw77" podStartSLOduration=1.389790268 podStartE2EDuration="4.065571815s" totalImagesPullingTime="2.675781547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:43:13.065324792 +0000 UTC m=+235.168078763" watchObservedRunningTime="2026-10-30 00:43:13.065571815 +0000 UTC m=+235.168325780"
Oct 30 00:43:13.161872 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.161847    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:13.341705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.341672    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53232519-8f82-4f9b-98d9-ebcfdab98045-kube-api-access-vgbst\" (UniqueName: \"kubernetes.io/projected/53232519-8f82-4f9b-98d9-ebcfdab98045-kube-api-access-vgbst\") pod \"53232519-8f82-4f9b-98d9-ebcfdab98045\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") "
Oct 30 00:43:13.341889 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.341807    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-util\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-util\") pod \"53232519-8f82-4f9b-98d9-ebcfdab98045\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") "
Oct 30 00:43:13.341889 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.341835    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-bundle\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-bundle\") pod \"53232519-8f82-4f9b-98d9-ebcfdab98045\" (UID: \"53232519-8f82-4f9b-98d9-ebcfdab98045\") "
Oct 30 00:43:13.342293 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.342269    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-bundle" pod "53232519-8f82-4f9b-98d9-ebcfdab98045" (UID: "53232519-8f82-4f9b-98d9-ebcfdab98045"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:43:13.343992 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.343964    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53232519-8f82-4f9b-98d9-ebcfdab98045-kube-api-access-vgbst" pod "53232519-8f82-4f9b-98d9-ebcfdab98045" (UID: "53232519-8f82-4f9b-98d9-ebcfdab98045"). InnerVolumeSpecName "kube-api-access-vgbst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:43:13.345923 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.345899    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-util" pod "53232519-8f82-4f9b-98d9-ebcfdab98045" (UID: "53232519-8f82-4f9b-98d9-ebcfdab98045"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:43:13.443246 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.443214    6011 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-util\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:43:13.443246 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.443238    6011 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/53232519-8f82-4f9b-98d9-ebcfdab98045-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:43:13.443246 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:13.443249    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgbst\" (UniqueName: \"kubernetes.io/projected/53232519-8f82-4f9b-98d9-ebcfdab98045-kube-api-access-vgbst\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:43:14.034398 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:14.034372    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr"
Oct 30 00:43:14.034752 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:14.034377    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/61b1ff68c7255ee5c5329fe19a5d96293a95788d9734a0a79829e8dc85vm9rr" event={"ID":"53232519-8f82-4f9b-98d9-ebcfdab98045","Type":"ContainerDied","Data":"c6a7e23099d492f1904f2541d176a3eb3211bce99ee7243f1ed18fc24a403d35"}
Oct 30 00:43:14.034752 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:14.034441    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c6a7e23099d492f1904f2541d176a3eb3211bce99ee7243f1ed18fc24a403d35"
Oct 30 00:43:17.763080 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763044    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-5tjbg"]
Oct 30 00:43:17.763495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763437    6011 cpu_manager.go:414] "containerMap: removing container" podUID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerName="pull"
Oct 30 00:43:17.763495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763452    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerName="pull"
Oct 30 00:43:17.763495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763475    6011 cpu_manager.go:414] "containerMap: removing container" podUID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerName="util"
Oct 30 00:43:17.763495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763481    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerName="util"
Oct 30 00:43:17.763495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763489    6011 cpu_manager.go:414] "containerMap: removing container" podUID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerName="extract"
Oct 30 00:43:17.763495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763494    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerName="extract"
Oct 30 00:43:17.763671 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.763563    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="53232519-8f82-4f9b-98d9-ebcfdab98045" containerName="extract"
Oct 30 00:43:17.765460 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.765441    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:17.785471 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.785447    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-5tjbg"]
Oct 30 00:43:17.796646 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.796625    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-n2tqr\""
Oct 30 00:43:17.880480 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.880450    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e03b342a-e7f9-4a46-b754-2d66d813c62a-bound-sa-token\") pod \"cert-manager-597b95cb84-5tjbg\" (UID: \"e03b342a-e7f9-4a46-b754-2d66d813c62a\") " pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:17.880618 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.880522    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbsxn\" (UniqueName: \"kubernetes.io/projected/e03b342a-e7f9-4a46-b754-2d66d813c62a-kube-api-access-fbsxn\") pod \"cert-manager-597b95cb84-5tjbg\" (UID: \"e03b342a-e7f9-4a46-b754-2d66d813c62a\") " pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:17.981349 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.981303    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e03b342a-e7f9-4a46-b754-2d66d813c62a-bound-sa-token\") pod \"cert-manager-597b95cb84-5tjbg\" (UID: \"e03b342a-e7f9-4a46-b754-2d66d813c62a\") " pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:17.981512 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:17.981415    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbsxn\" (UniqueName: \"kubernetes.io/projected/e03b342a-e7f9-4a46-b754-2d66d813c62a-kube-api-access-fbsxn\") pod \"cert-manager-597b95cb84-5tjbg\" (UID: \"e03b342a-e7f9-4a46-b754-2d66d813c62a\") " pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:18.000950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:18.000919    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e03b342a-e7f9-4a46-b754-2d66d813c62a-bound-sa-token\") pod \"cert-manager-597b95cb84-5tjbg\" (UID: \"e03b342a-e7f9-4a46-b754-2d66d813c62a\") " pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:18.001073 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:18.001034    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbsxn\" (UniqueName: \"kubernetes.io/projected/e03b342a-e7f9-4a46-b754-2d66d813c62a-kube-api-access-fbsxn\") pod \"cert-manager-597b95cb84-5tjbg\" (UID: \"e03b342a-e7f9-4a46-b754-2d66d813c62a\") " pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:18.089466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:18.089442    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-n2tqr\""
Oct 30 00:43:18.096939 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:18.096921    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-5tjbg"
Oct 30 00:43:18.386339 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:18.386259    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-5tjbg"]
Oct 30 00:43:18.389677 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:18.389646    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode03b342a_e7f9_4a46_b754_2d66d813c62a.slice/crio-cdb4839297d38b9c99c1c08da815029dcad6b4f38454edba6a952c0673c9b9ea WatchSource:0}: Error finding container cdb4839297d38b9c99c1c08da815029dcad6b4f38454edba6a952c0673c9b9ea: Status 404 returned error can't find the container with id cdb4839297d38b9c99c1c08da815029dcad6b4f38454edba6a952c0673c9b9ea
Oct 30 00:43:19.054497 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:19.054460    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-5tjbg" event={"ID":"e03b342a-e7f9-4a46-b754-2d66d813c62a","Type":"ContainerStarted","Data":"5291d7c367c0c59d3eda7cc92167a60e73ec88ee973b851d6909d225ae19ef0a"}
Oct 30 00:43:19.054497 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:19.054500    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-5tjbg" event={"ID":"e03b342a-e7f9-4a46-b754-2d66d813c62a","Type":"ContainerStarted","Data":"cdb4839297d38b9c99c1c08da815029dcad6b4f38454edba6a952c0673c9b9ea"}
Oct 30 00:43:19.092033 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:19.091983    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-5tjbg" podStartSLOduration=2.091971028 podStartE2EDuration="2.091971028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:43:19.091361903 +0000 UTC m=+241.194115887" watchObservedRunningTime="2026-10-30 00:43:19.091971028 +0000 UTC m=+241.194724993"
Oct 30 00:43:20.019291 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.019260    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"]
Oct 30 00:43:20.021381 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.021366    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.035200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.035142    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"]
Oct 30 00:43:20.046630 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.046604    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bkskl\""
Oct 30 00:43:20.046750 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.046674    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Oct 30 00:43:20.051395 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.051379    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Oct 30 00:43:20.100521 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.100486    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d-tmp\") pod \"openshift-lws-operator-8465994787-nnlhw\" (UID: \"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.100945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.100583    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hmrj\" (UniqueName: \"kubernetes.io/projected/73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d-kube-api-access-5hmrj\") pod \"openshift-lws-operator-8465994787-nnlhw\" (UID: \"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.201517 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.201472    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d-tmp\") pod \"openshift-lws-operator-8465994787-nnlhw\" (UID: \"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.201714 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.201574    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hmrj\" (UniqueName: \"kubernetes.io/projected/73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d-kube-api-access-5hmrj\") pod \"openshift-lws-operator-8465994787-nnlhw\" (UID: \"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.202055 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.202013    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d-tmp\") pod \"openshift-lws-operator-8465994787-nnlhw\" (UID: \"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.222008 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.221937    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hmrj\" (UniqueName: \"kubernetes.io/projected/73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d-kube-api-access-5hmrj\") pod \"openshift-lws-operator-8465994787-nnlhw\" (UID: \"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d\") " pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.332544 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.332510    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"
Oct 30 00:43:20.622232 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:20.622130    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw"]
Oct 30 00:43:20.625207 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:20.625155    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73d5ac96_6743_4c1e_a98a_67b2fc5f8b1d.slice/crio-f48e1cedaad4c64bd8c6b75aa20fa847c251d9c8a0f5a1b9de926850943ba0ef WatchSource:0}: Error finding container f48e1cedaad4c64bd8c6b75aa20fa847c251d9c8a0f5a1b9de926850943ba0ef: Status 404 returned error can't find the container with id f48e1cedaad4c64bd8c6b75aa20fa847c251d9c8a0f5a1b9de926850943ba0ef
Oct 30 00:43:21.063927 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:21.063887    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw" event={"ID":"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d","Type":"ContainerStarted","Data":"f48e1cedaad4c64bd8c6b75aa20fa847c251d9c8a0f5a1b9de926850943ba0ef"}
Oct 30 00:43:23.075311 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:23.075277    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw" event={"ID":"73d5ac96-6743-4c1e-a98a-67b2fc5f8b1d","Type":"ContainerStarted","Data":"eb60e23701962795f2e2ca126e2d74dc5301c71939ac95ca927d56aa9617c495"}
Oct 30 00:43:23.117114 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:23.117064    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-8465994787-nnlhw" podStartSLOduration=2.212545887 podStartE2EDuration="4.117048574s" totalImagesPullingTime="1.904502687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:43:23.11578226 +0000 UTC m=+245.218536231" watchObservedRunningTime="2026-10-30 00:43:23.117048574 +0000 UTC m=+245.219802545"
Oct 30 00:43:27.167345 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.167288    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"]
Oct 30 00:43:27.169848 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.169832    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.198419 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.198395    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 00:43:27.198419 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.198398    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 00:43:27.198592 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.198461    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e644421527c169d876fdb59235c927986d6bd2-9d2815fc-dockercfg-pz8dk\""
Oct 30 00:43:27.199624 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.199600    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"]
Oct 30 00:43:27.251269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.251218    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh9k6\" (UniqueName: \"kubernetes.io/projected/d07e25a0-c3ef-490e-bf50-8063e12c17a0-kube-api-access-zh9k6\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.251269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.251276    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.251484 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.251376    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.351952 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.351910    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.352156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.351962    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.352156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.352065    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh9k6\" (UniqueName: \"kubernetes.io/projected/d07e25a0-c3ef-490e-bf50-8063e12c17a0-kube-api-access-zh9k6\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.352351 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.352331    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-util\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.352415 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.352363    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-bundle\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.371405 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.371371    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh9k6\" (UniqueName: \"kubernetes.io/projected/d07e25a0-c3ef-490e-bf50-8063e12c17a0-kube-api-access-zh9k6\") pod \"e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") " pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.482427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.482336    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:27.773891 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:27.773819    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"]
Oct 30 00:43:27.776654 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:27.776622    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd07e25a0_c3ef_490e_bf50_8063e12c17a0.slice/crio-d1db553eaeed05672d04af9ef90a5864916070dfea96a07477397f68385a1b4d WatchSource:0}: Error finding container d1db553eaeed05672d04af9ef90a5864916070dfea96a07477397f68385a1b4d: Status 404 returned error can't find the container with id d1db553eaeed05672d04af9ef90a5864916070dfea96a07477397f68385a1b4d
Oct 30 00:43:27.908370 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:43:27.908328    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:43:28.095462 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:28.095424    6011 generic.go:356] "Generic (PLEG): container finished" podID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerID="3e3f3cf82dbc86fe88e276cb3ce83b5c6ef198a6dba2302820fc2768c1041d95" exitCode=0
Oct 30 00:43:28.095662 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:28.095524    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b" event={"ID":"d07e25a0-c3ef-490e-bf50-8063e12c17a0","Type":"ContainerDied","Data":"3e3f3cf82dbc86fe88e276cb3ce83b5c6ef198a6dba2302820fc2768c1041d95"}
Oct 30 00:43:28.095662 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:28.095587    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b" event={"ID":"d07e25a0-c3ef-490e-bf50-8063e12c17a0","Type":"ContainerStarted","Data":"d1db553eaeed05672d04af9ef90a5864916070dfea96a07477397f68385a1b4d"}
Oct 30 00:43:29.101664 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:29.101625    6011 generic.go:356] "Generic (PLEG): container finished" podID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerID="b6f20a62cb07c9d8dc075e368e3c8115156bc6def4ac7f0acd56f753f79e89c6" exitCode=0
Oct 30 00:43:29.102135 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:29.101704    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b" event={"ID":"d07e25a0-c3ef-490e-bf50-8063e12c17a0","Type":"ContainerDied","Data":"b6f20a62cb07c9d8dc075e368e3c8115156bc6def4ac7f0acd56f753f79e89c6"}
Oct 30 00:43:30.108104 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:30.108067    6011 generic.go:356] "Generic (PLEG): container finished" podID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerID="13209bfcfc35b3a65e3f35bc6b369d67ebd4b81951184ccbf529e79e630f3f7b" exitCode=0
Oct 30 00:43:30.108513 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:30.108129    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b" event={"ID":"d07e25a0-c3ef-490e-bf50-8063e12c17a0","Type":"ContainerDied","Data":"13209bfcfc35b3a65e3f35bc6b369d67ebd4b81951184ccbf529e79e630f3f7b"}
Oct 30 00:43:31.239537 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.239512    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:31.288110 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.288080    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-util\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-util\") pod \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") "
Oct 30 00:43:31.288327 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.288143    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-bundle\") pod \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") "
Oct 30 00:43:31.288327 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.288228    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d07e25a0-c3ef-490e-bf50-8063e12c17a0-kube-api-access-zh9k6\" (UniqueName: \"kubernetes.io/projected/d07e25a0-c3ef-490e-bf50-8063e12c17a0-kube-api-access-zh9k6\") pod \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\" (UID: \"d07e25a0-c3ef-490e-bf50-8063e12c17a0\") "
Oct 30 00:43:31.288794 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.288759    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-bundle" pod "d07e25a0-c3ef-490e-bf50-8063e12c17a0" (UID: "d07e25a0-c3ef-490e-bf50-8063e12c17a0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:43:31.290552 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.290522    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d07e25a0-c3ef-490e-bf50-8063e12c17a0-kube-api-access-zh9k6" pod "d07e25a0-c3ef-490e-bf50-8063e12c17a0" (UID: "d07e25a0-c3ef-490e-bf50-8063e12c17a0"). InnerVolumeSpecName "kube-api-access-zh9k6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:43:31.293496 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.293468    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-util" pod "d07e25a0-c3ef-490e-bf50-8063e12c17a0" (UID: "d07e25a0-c3ef-490e-bf50-8063e12c17a0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:43:31.389343 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.389263    6011 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-util\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:43:31.389343 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.389291    6011 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d07e25a0-c3ef-490e-bf50-8063e12c17a0-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:43:31.389343 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:31.389301    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh9k6\" (UniqueName: \"kubernetes.io/projected/d07e25a0-c3ef-490e-bf50-8063e12c17a0-kube-api-access-zh9k6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:43:32.118275 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:32.118241    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b"
Oct 30 00:43:32.118459 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:32.118236    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e644421527c169d876fdb59235c927986d6bd2881f72d188068392be71hjh8b" event={"ID":"d07e25a0-c3ef-490e-bf50-8063e12c17a0","Type":"ContainerDied","Data":"d1db553eaeed05672d04af9ef90a5864916070dfea96a07477397f68385a1b4d"}
Oct 30 00:43:32.118459 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:32.118364    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1db553eaeed05672d04af9ef90a5864916070dfea96a07477397f68385a1b4d"
Oct 30 00:43:37.287795 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.287752    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"]
Oct 30 00:43:37.288318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.288189    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerName="util"
Oct 30 00:43:37.288318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.288204    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerName="util"
Oct 30 00:43:37.288318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.288260    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerName="pull"
Oct 30 00:43:37.288318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.288266    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerName="pull"
Oct 30 00:43:37.288318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.288279    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerName="extract"
Oct 30 00:43:37.288318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.288284    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerName="extract"
Oct 30 00:43:37.288514 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.288357    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="d07e25a0-c3ef-490e-bf50-8063e12c17a0" containerName="extract"
Oct 30 00:43:37.290687 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.290671    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.306741 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.306710    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"]
Oct 30 00:43:37.344411 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.344372    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aed71d7a-a307-474d-8532-87e110b81265-tmp\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.344593 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.344435    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-webhook-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.344593 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.344496    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-metrics-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.344593 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.344583    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.344694 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.344622    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aed71d7a-a307-474d-8532-87e110b81265-manifests\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.344694 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.344641    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmqtp\" (UniqueName: \"kubernetes.io/projected/aed71d7a-a307-474d-8532-87e110b81265-kube-api-access-rmqtp\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.395469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.395433    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 00:43:37.395653 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.395485    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 00:43:37.396127 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.396103    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 00:43:37.396299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.396164    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\""
Oct 30 00:43:37.396299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.396194    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 00:43:37.396299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.396245    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-f4n97\""
Oct 30 00:43:37.445966 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.445932    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-webhook-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.446083 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.445970    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-metrics-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.446083 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.446001    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.446083 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.446032    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aed71d7a-a307-474d-8532-87e110b81265-manifests\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.446083 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.446059    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmqtp\" (UniqueName: \"kubernetes.io/projected/aed71d7a-a307-474d-8532-87e110b81265-kube-api-access-rmqtp\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.446305 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.446091    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aed71d7a-a307-474d-8532-87e110b81265-tmp\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.446551 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.446522    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/aed71d7a-a307-474d-8532-87e110b81265-manifests\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.448720 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.448693    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aed71d7a-a307-474d-8532-87e110b81265-tmp\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.448812 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.448744    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-webhook-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.448908 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.448892    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-metrics-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.448945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.448922    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/aed71d7a-a307-474d-8532-87e110b81265-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.467032 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.467008    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmqtp\" (UniqueName: \"kubernetes.io/projected/aed71d7a-a307-474d-8532-87e110b81265-kube-api-access-rmqtp\") pod \"opendatahub-operator-controller-manager-8cc7fc69d-vbxkf\" (UID: \"aed71d7a-a307-474d-8532-87e110b81265\") " pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.603330 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.603226    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:37.950644 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:37.950612    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaed71d7a_a307_474d_8532_87e110b81265.slice/crio-242488f67e4a0d78edabcdefd05751be8ca48a4822def336c0d0d7e0dc8b3266 WatchSource:0}: Error finding container 242488f67e4a0d78edabcdefd05751be8ca48a4822def336c0d0d7e0dc8b3266: Status 404 returned error can't find the container with id 242488f67e4a0d78edabcdefd05751be8ca48a4822def336c0d0d7e0dc8b3266
Oct 30 00:43:37.960221 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:37.960197    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"]
Oct 30 00:43:38.145740 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:38.145703    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf" event={"ID":"aed71d7a-a307-474d-8532-87e110b81265","Type":"ContainerStarted","Data":"242488f67e4a0d78edabcdefd05751be8ca48a4822def336c0d0d7e0dc8b3266"}
Oct 30 00:43:41.162749 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:41.162665    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf" event={"ID":"aed71d7a-a307-474d-8532-87e110b81265","Type":"ContainerStarted","Data":"edde94a28fadfdaa5a1ab2cdc7ae221bb35668f97f0fb2f0d55aacbe3ecffb3a"}
Oct 30 00:43:42.169004 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:42.168970    6011 generic.go:356] "Generic (PLEG): container finished" podID="aed71d7a-a307-474d-8532-87e110b81265" containerID="edde94a28fadfdaa5a1ab2cdc7ae221bb35668f97f0fb2f0d55aacbe3ecffb3a" exitCode=0
Oct 30 00:43:42.169427 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:42.169065    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf" event={"ID":"aed71d7a-a307-474d-8532-87e110b81265","Type":"ContainerDied","Data":"edde94a28fadfdaa5a1ab2cdc7ae221bb35668f97f0fb2f0d55aacbe3ecffb3a"}
Oct 30 00:43:43.178215 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:43.178162    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf" event={"ID":"aed71d7a-a307-474d-8532-87e110b81265","Type":"ContainerStarted","Data":"eb938629cd72a7dab62202d88e32afb9f429478666942050c24104762dd9ac45"}
Oct 30 00:43:43.178677 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:43.178302    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:43.260137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:43.260087    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf" podStartSLOduration=2.5012254929999997 podStartE2EDuration="6.260073959s" totalImagesPullingTime="3.193294592s" totalInitContainerRuntime="565.553874ms" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:43:43.247727265 +0000 UTC m=+265.350481235" watchObservedRunningTime="2026-10-30 00:43:43.260073959 +0000 UTC m=+265.362827931"
Oct 30 00:43:54.185115 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:54.185083    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8cc7fc69d-vbxkf"
Oct 30 00:43:57.094906 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.094875    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"]
Oct 30 00:43:57.105064 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.105032    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.118576 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.118547    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"]
Oct 30 00:43:57.129443 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.129408    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 00:43:57.129443 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.129408    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 00:43:57.209049 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.209018    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 00:43:57.209271 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.209223    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 00:43:57.209271 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.209259    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-s98tb\""
Oct 30 00:43:57.236719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.236690    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-tls-certs\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.236889 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.236726    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx6ph\" (UniqueName: \"kubernetes.io/projected/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-kube-api-access-fx6ph\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.236889 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.236818    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-tmp\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.337913 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.337873    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-tmp\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.338107 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.337957    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-tls-certs\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.338107 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.337983    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx6ph\" (UniqueName: \"kubernetes.io/projected/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-kube-api-access-fx6ph\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.340432 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.340407    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-tmp\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.340669 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.340651    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-tls-certs\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.357147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.357058    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx6ph\" (UniqueName: \"kubernetes.io/projected/dcad1ee6-9300-4dd7-98fd-6a05d3a6b867-kube-api-access-fx6ph\") pod \"kube-auth-proxy-7597b65cbf-stpb4\" (UID: \"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867\") " pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.419058 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.419018    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"
Oct 30 00:43:57.715370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:57.715142    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4"]
Oct 30 00:43:57.718503 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:57.718477    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcad1ee6_9300_4dd7_98fd_6a05d3a6b867.slice/crio-6ff6806d8a958b144cdf3fbc764c5d1b41e292abbac6a98236d288a9ac6cfb66 WatchSource:0}: Error finding container 6ff6806d8a958b144cdf3fbc764c5d1b41e292abbac6a98236d288a9ac6cfb66: Status 404 returned error can't find the container with id 6ff6806d8a958b144cdf3fbc764c5d1b41e292abbac6a98236d288a9ac6cfb66
Oct 30 00:43:57.909274 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:43:57.909238    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:43:58.246148 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:58.246105    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4" event={"ID":"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867","Type":"ContainerStarted","Data":"6ff6806d8a958b144cdf3fbc764c5d1b41e292abbac6a98236d288a9ac6cfb66"}
Oct 30 00:43:59.188061 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.188023    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"]
Oct 30 00:43:59.196957 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.196874    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.224670 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.223412    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Oct 30 00:43:59.236957 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.236925    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"]
Oct 30 00:43:59.306934 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.306894    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Oct 30 00:43:59.311816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.311791    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bdjc8\""
Oct 30 00:43:59.311978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.311819    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Oct 30 00:43:59.365012 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.364981    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d6133a9e-fd60-490a-bfeb-37b83aad8636-manager-config\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.365206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.365053    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6133a9e-fd60-490a-bfeb-37b83aad8636-cert\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.365266 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.365243    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdnkj\" (UniqueName: \"kubernetes.io/projected/d6133a9e-fd60-490a-bfeb-37b83aad8636-kube-api-access-jdnkj\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.365315 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.365285    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6133a9e-fd60-490a-bfeb-37b83aad8636-metrics-cert\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.466651 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.466568    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6133a9e-fd60-490a-bfeb-37b83aad8636-cert\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.466651 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.466637    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdnkj\" (UniqueName: \"kubernetes.io/projected/d6133a9e-fd60-490a-bfeb-37b83aad8636-kube-api-access-jdnkj\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.466869 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.466664    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6133a9e-fd60-490a-bfeb-37b83aad8636-metrics-cert\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.466869 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.466723    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d6133a9e-fd60-490a-bfeb-37b83aad8636-manager-config\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.467562 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.467492    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d6133a9e-fd60-490a-bfeb-37b83aad8636-manager-config\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.469758 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.469737    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6133a9e-fd60-490a-bfeb-37b83aad8636-cert\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.471289 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.471269    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d6133a9e-fd60-490a-bfeb-37b83aad8636-metrics-cert\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.490297 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.490264    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdnkj\" (UniqueName: \"kubernetes.io/projected/d6133a9e-fd60-490a-bfeb-37b83aad8636-kube-api-access-jdnkj\") pod \"lws-controller-manager-7459cbd475-slnhc\" (UID: \"d6133a9e-fd60-490a-bfeb-37b83aad8636\") " pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.515989 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.515960    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:43:59.819299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:43:59.819262    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"]
Oct 30 00:43:59.822299 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:43:59.822267    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6133a9e_fd60_490a_bfeb_37b83aad8636.slice/crio-2f36ddc9fcd6b7559db12e08950d35c4001ed450664bf8eb4a734004e85bc459 WatchSource:0}: Error finding container 2f36ddc9fcd6b7559db12e08950d35c4001ed450664bf8eb4a734004e85bc459: Status 404 returned error can't find the container with id 2f36ddc9fcd6b7559db12e08950d35c4001ed450664bf8eb4a734004e85bc459
Oct 30 00:44:00.258371 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:44:00.258327    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc" event={"ID":"d6133a9e-fd60-490a-bfeb-37b83aad8636","Type":"ContainerStarted","Data":"2f36ddc9fcd6b7559db12e08950d35c4001ed450664bf8eb4a734004e85bc459"}
Oct 30 00:44:00.259828 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:44:00.259801    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4" event={"ID":"dcad1ee6-9300-4dd7-98fd-6a05d3a6b867","Type":"ContainerStarted","Data":"792a5a6dec6e2b85a1239d5d65e891aad9211dfbe20b85320d27f204e78137b9"}
Oct 30 00:44:00.298719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:44:00.298665    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7597b65cbf-stpb4" podStartSLOduration=0.889235717 podStartE2EDuration="3.298651621s" totalImagesPullingTime="2.409415904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:44:00.29686522 +0000 UTC m=+282.399619202" watchObservedRunningTime="2026-10-30 00:44:00.298651621 +0000 UTC m=+282.401405591"
Oct 30 00:44:08.300711 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:44:08.300666    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc" event={"ID":"d6133a9e-fd60-490a-bfeb-37b83aad8636","Type":"ContainerStarted","Data":"d0c970293e648d2300a70a6c57499062b64e41aa84c8507d81917cd66d3faeb6"}
Oct 30 00:44:08.301065 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:44:08.300822    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:44:08.339627 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:44:08.339575    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc" podStartSLOduration=0.960445678 podStartE2EDuration="9.339561823s" totalImagesPullingTime="8.379116145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:43:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:44:08.338904635 +0000 UTC m=+290.441658606" watchObservedRunningTime="2026-10-30 00:44:08.339561823 +0000 UTC m=+290.442315793"
Oct 30 00:44:19.307851 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:44:19.307824    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7459cbd475-slnhc"
Oct 30 00:44:27.908864 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:44:27.908829    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:44:57.909431 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:44:57.909394    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:45:27.909441 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:45:27.909355    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:45:52.278399 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.278368    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"]
Oct 30 00:45:52.282117 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.282099    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.308905 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.308873    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"6541d93aaee335d6227a8778ab4e840bbe7f52-f98fbcf8-dockercfg-8wtjz\""
Oct 30 00:45:52.309155 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.308958    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 00:45:52.310145 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.310111    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"]
Oct 30 00:45:52.351731 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.351694    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 00:45:52.366728 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.366691    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"]
Oct 30 00:45:52.370431 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.370409    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.385924 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.385894    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"da112bd609bff87f7bacd2ba941225d7cff15c-bd73b2fc-dockercfg-zv29w\""
Oct 30 00:45:52.387561 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.387533    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"]
Oct 30 00:45:52.419107 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.419068    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.419107 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.419108    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.419351 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.419215    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxkbk\" (UniqueName: \"kubernetes.io/projected/3121e65f-0745-44fa-b848-8df1148c4421-kube-api-access-kxkbk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.466100 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.466068    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"]
Oct 30 00:45:52.469820 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.469800    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.487244 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.487214    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3-b83c8cba-dockercfg-fqdkl\""
Oct 30 00:45:52.488489 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.488470    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"]
Oct 30 00:45:52.520474 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.520436    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.520669 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.520491    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tmqh\" (UniqueName: \"kubernetes.io/projected/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-kube-api-access-8tmqh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.520669 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.520638    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.520758 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.520677    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.520758 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.520709    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.520837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.520790    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxkbk\" (UniqueName: \"kubernetes.io/projected/3121e65f-0745-44fa-b848-8df1148c4421-kube-api-access-kxkbk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.521027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.521013    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.521096 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.521076    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.539963 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.539936    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxkbk\" (UniqueName: \"kubernetes.io/projected/3121e65f-0745-44fa-b848-8df1148c4421-kube-api-access-kxkbk\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.561447 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.561408    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"]
Oct 30 00:45:52.565424 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.565406    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.582645 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.582617    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"]
Oct 30 00:45:52.587902 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.587872    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-mgkdq\""
Oct 30 00:45:52.601858 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.601833    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:52.621898 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.621858    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.622092 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.621937    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tmqh\" (UniqueName: \"kubernetes.io/projected/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-kube-api-access-8tmqh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.622092 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.621984    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v8zp\" (UniqueName: \"kubernetes.io/projected/323ba850-3f40-459e-9da4-078f0eaa2730-kube-api-access-2v8zp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.622092 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.622032    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.622299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.622100    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.622299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.622205    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.622399 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.622340    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.622446 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.622414    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.642994 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.642956    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tmqh\" (UniqueName: \"kubernetes.io/projected/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-kube-api-access-8tmqh\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.687598 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.687561    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:52.723653 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.723613    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.723844 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.723669    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v8zp\" (UniqueName: \"kubernetes.io/projected/323ba850-3f40-459e-9da4-078f0eaa2730-kube-api-access-2v8zp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.723844 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.723702    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.723965 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.723903    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.723965 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.723956    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.724062 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.723988    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtc6\" (UniqueName: \"kubernetes.io/projected/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-kube-api-access-4xtc6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.724343 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.724326    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.724397 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.724368    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.746964 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.746763    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v8zp\" (UniqueName: \"kubernetes.io/projected/323ba850-3f40-459e-9da4-078f0eaa2730-kube-api-access-2v8zp\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.787022 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.786982    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:52.824486 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.824449    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtc6\" (UniqueName: \"kubernetes.io/projected/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-kube-api-access-4xtc6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.824659 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.824521    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.824659 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.824569    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.825028 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.825005    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.825074 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.825032    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.845384 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.845354    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtc6\" (UniqueName: \"kubernetes.io/projected/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-kube-api-access-4xtc6\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.883962 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.883927    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:52.900031 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.900002    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"]
Oct 30 00:45:52.902253 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:45:52.902221    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3121e65f_0745_44fa_b848_8df1148c4421.slice/crio-c4657bd8c4a2bd5eada7002d77f33e88aa2441832ed45e66e87eb899698cc29f WatchSource:0}: Error finding container c4657bd8c4a2bd5eada7002d77f33e88aa2441832ed45e66e87eb899698cc29f: Status 404 returned error can't find the container with id c4657bd8c4a2bd5eada7002d77f33e88aa2441832ed45e66e87eb899698cc29f
Oct 30 00:45:52.983562 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:52.983532    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"]
Oct 30 00:45:52.986319 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:45:52.986291    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c27010f_e26c_4c22_8ec8_d10dbf0e3084.slice/crio-8e655e1d808fe3dc041e8cc99f438153c70d43358715e0235ca5d92418d04523 WatchSource:0}: Error finding container 8e655e1d808fe3dc041e8cc99f438153c70d43358715e0235ca5d92418d04523: Status 404 returned error can't find the container with id 8e655e1d808fe3dc041e8cc99f438153c70d43358715e0235ca5d92418d04523
Oct 30 00:45:53.083026 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.082448    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"]
Oct 30 00:45:53.113967 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:45:53.113926    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod323ba850_3f40_459e_9da4_078f0eaa2730.slice/crio-5b1dd857fbadfb77725c90c216e3b3e882bc2439ac8bb9a9e0dde4fb47642055 WatchSource:0}: Error finding container 5b1dd857fbadfb77725c90c216e3b3e882bc2439ac8bb9a9e0dde4fb47642055: Status 404 returned error can't find the container with id 5b1dd857fbadfb77725c90c216e3b3e882bc2439ac8bb9a9e0dde4fb47642055
Oct 30 00:45:53.183201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.183137    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"]
Oct 30 00:45:53.186828 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:45:53.186794    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04663262_a0ce_4647_85cb_8e8d9e8a5bb2.slice/crio-6394ecce01fa7e42634881699cf4131b0952b9dad601c0c58496070c8a3a4993 WatchSource:0}: Error finding container 6394ecce01fa7e42634881699cf4131b0952b9dad601c0c58496070c8a3a4993: Status 404 returned error can't find the container with id 6394ecce01fa7e42634881699cf4131b0952b9dad601c0c58496070c8a3a4993
Oct 30 00:45:53.778795 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.778702    6011 generic.go:356] "Generic (PLEG): container finished" podID="323ba850-3f40-459e-9da4-078f0eaa2730" containerID="ab0924a3e5c62ad2d9a9358395edab37af850d9f288b832346d5cfe43d18d3da" exitCode=0
Oct 30 00:45:53.778795 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.778778    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh" event={"ID":"323ba850-3f40-459e-9da4-078f0eaa2730","Type":"ContainerDied","Data":"ab0924a3e5c62ad2d9a9358395edab37af850d9f288b832346d5cfe43d18d3da"}
Oct 30 00:45:53.779418 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.778833    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh" event={"ID":"323ba850-3f40-459e-9da4-078f0eaa2730","Type":"ContainerStarted","Data":"5b1dd857fbadfb77725c90c216e3b3e882bc2439ac8bb9a9e0dde4fb47642055"}
Oct 30 00:45:53.780481 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.780454    6011 generic.go:356] "Generic (PLEG): container finished" podID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerID="546c861d7ccef50f488679d980b08d3bac2ff644a0dacdd416fe09cfa66b1dc8" exitCode=0
Oct 30 00:45:53.780564 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.780541    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc" event={"ID":"4c27010f-e26c-4c22-8ec8-d10dbf0e3084","Type":"ContainerDied","Data":"546c861d7ccef50f488679d980b08d3bac2ff644a0dacdd416fe09cfa66b1dc8"}
Oct 30 00:45:53.780616 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.780585    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc" event={"ID":"4c27010f-e26c-4c22-8ec8-d10dbf0e3084","Type":"ContainerStarted","Data":"8e655e1d808fe3dc041e8cc99f438153c70d43358715e0235ca5d92418d04523"}
Oct 30 00:45:53.782225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.782203    6011 generic.go:356] "Generic (PLEG): container finished" podID="3121e65f-0745-44fa-b848-8df1148c4421" containerID="7865dbfee7ad383c29724552ebff51ba49bef1504ccf77ee03f8c31e6161b58a" exitCode=0
Oct 30 00:45:53.782318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.782280    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5" event={"ID":"3121e65f-0745-44fa-b848-8df1148c4421","Type":"ContainerDied","Data":"7865dbfee7ad383c29724552ebff51ba49bef1504ccf77ee03f8c31e6161b58a"}
Oct 30 00:45:53.782372 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.782332    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5" event={"ID":"3121e65f-0745-44fa-b848-8df1148c4421","Type":"ContainerStarted","Data":"c4657bd8c4a2bd5eada7002d77f33e88aa2441832ed45e66e87eb899698cc29f"}
Oct 30 00:45:53.784047 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.783920    6011 generic.go:356] "Generic (PLEG): container finished" podID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerID="fdb73e39479f270cda067e376e3c80e2edff97a03aa0978ad1532d9a0f75978f" exitCode=0
Oct 30 00:45:53.784047 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.783965    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5" event={"ID":"04663262-a0ce-4647-85cb-8e8d9e8a5bb2","Type":"ContainerDied","Data":"fdb73e39479f270cda067e376e3c80e2edff97a03aa0978ad1532d9a0f75978f"}
Oct 30 00:45:53.784047 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:53.784007    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5" event={"ID":"04663262-a0ce-4647-85cb-8e8d9e8a5bb2","Type":"ContainerStarted","Data":"6394ecce01fa7e42634881699cf4131b0952b9dad601c0c58496070c8a3a4993"}
Oct 30 00:45:54.793345 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:54.793310    6011 generic.go:356] "Generic (PLEG): container finished" podID="323ba850-3f40-459e-9da4-078f0eaa2730" containerID="743026b7da724fb32411ae9016620ae3602f48f767c6777c299732739bfab27f" exitCode=0
Oct 30 00:45:54.793724 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:54.793396    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh" event={"ID":"323ba850-3f40-459e-9da4-078f0eaa2730","Type":"ContainerDied","Data":"743026b7da724fb32411ae9016620ae3602f48f767c6777c299732739bfab27f"}
Oct 30 00:45:55.801542 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.801509    6011 generic.go:356] "Generic (PLEG): container finished" podID="323ba850-3f40-459e-9da4-078f0eaa2730" containerID="6fc80d7a3c4b5af776a9644d7a1250cc1d45a68eb3c424a4567efadf9a72c749" exitCode=0
Oct 30 00:45:55.801967 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.801592    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh" event={"ID":"323ba850-3f40-459e-9da4-078f0eaa2730","Type":"ContainerDied","Data":"6fc80d7a3c4b5af776a9644d7a1250cc1d45a68eb3c424a4567efadf9a72c749"}
Oct 30 00:45:55.803469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.803446    6011 generic.go:356] "Generic (PLEG): container finished" podID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerID="014df8d032bfb791904604727c734744436c45692ceeb6ee1c34a457b5f195ae" exitCode=0
Oct 30 00:45:55.803631 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.803509    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc" event={"ID":"4c27010f-e26c-4c22-8ec8-d10dbf0e3084","Type":"ContainerDied","Data":"014df8d032bfb791904604727c734744436c45692ceeb6ee1c34a457b5f195ae"}
Oct 30 00:45:55.805622 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.805605    6011 generic.go:356] "Generic (PLEG): container finished" podID="3121e65f-0745-44fa-b848-8df1148c4421" containerID="a179e65059a289f0c86a7baa0bb1e0e7ea9d0c496927309a2a0aed420e33a339" exitCode=0
Oct 30 00:45:55.805746 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.805680    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5" event={"ID":"3121e65f-0745-44fa-b848-8df1148c4421","Type":"ContainerDied","Data":"a179e65059a289f0c86a7baa0bb1e0e7ea9d0c496927309a2a0aed420e33a339"}
Oct 30 00:45:55.807820 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.807798    6011 generic.go:356] "Generic (PLEG): container finished" podID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerID="6d22a7f3059c1d66e764cc8f6ebe0ed49ecec31ce068562e673db5e4d17c14b8" exitCode=0
Oct 30 00:45:55.807917 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:55.807878    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5" event={"ID":"04663262-a0ce-4647-85cb-8e8d9e8a5bb2","Type":"ContainerDied","Data":"6d22a7f3059c1d66e764cc8f6ebe0ed49ecec31ce068562e673db5e4d17c14b8"}
Oct 30 00:45:56.821034 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:56.820940    6011 generic.go:356] "Generic (PLEG): container finished" podID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerID="4655c5676c287d6d89af273d119cd1a232fca9aa4e54bfa09b558a61487766cc" exitCode=0
Oct 30 00:45:56.821492 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:56.821028    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc" event={"ID":"4c27010f-e26c-4c22-8ec8-d10dbf0e3084","Type":"ContainerDied","Data":"4655c5676c287d6d89af273d119cd1a232fca9aa4e54bfa09b558a61487766cc"}
Oct 30 00:45:56.823030 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:56.823009    6011 generic.go:356] "Generic (PLEG): container finished" podID="3121e65f-0745-44fa-b848-8df1148c4421" containerID="7c063e3baee2db4eb5895452dc0f7f71e634a86b73117e1b45bbfc6c1b1b83de" exitCode=0
Oct 30 00:45:56.823146 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:56.823090    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5" event={"ID":"3121e65f-0745-44fa-b848-8df1148c4421","Type":"ContainerDied","Data":"7c063e3baee2db4eb5895452dc0f7f71e634a86b73117e1b45bbfc6c1b1b83de"}
Oct 30 00:45:56.825036 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:56.825015    6011 generic.go:356] "Generic (PLEG): container finished" podID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerID="bee2e21f7ca7a6dd8ae1f9350cfbe5a6d17ccf6d35a582e4a7b337d9558b036f" exitCode=0
Oct 30 00:45:56.825137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:56.825091    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5" event={"ID":"04663262-a0ce-4647-85cb-8e8d9e8a5bb2","Type":"ContainerDied","Data":"bee2e21f7ca7a6dd8ae1f9350cfbe5a6d17ccf6d35a582e4a7b337d9558b036f"}
Oct 30 00:45:56.959703 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:56.959677    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:57.064095 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.064061    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-util\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-util\") pod \"323ba850-3f40-459e-9da4-078f0eaa2730\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") "
Oct 30 00:45:57.064271 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.064104    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-bundle\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-bundle\") pod \"323ba850-3f40-459e-9da4-078f0eaa2730\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") "
Oct 30 00:45:57.064271 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.064154    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/323ba850-3f40-459e-9da4-078f0eaa2730-kube-api-access-2v8zp\" (UniqueName: \"kubernetes.io/projected/323ba850-3f40-459e-9da4-078f0eaa2730-kube-api-access-2v8zp\") pod \"323ba850-3f40-459e-9da4-078f0eaa2730\" (UID: \"323ba850-3f40-459e-9da4-078f0eaa2730\") "
Oct 30 00:45:57.064607 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.064574    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-bundle" pod "323ba850-3f40-459e-9da4-078f0eaa2730" (UID: "323ba850-3f40-459e-9da4-078f0eaa2730"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:57.066514 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.066483    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/323ba850-3f40-459e-9da4-078f0eaa2730-kube-api-access-2v8zp" pod "323ba850-3f40-459e-9da4-078f0eaa2730" (UID: "323ba850-3f40-459e-9da4-078f0eaa2730"). InnerVolumeSpecName "kube-api-access-2v8zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:45:57.069359 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.069336    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-util" pod "323ba850-3f40-459e-9da4-078f0eaa2730" (UID: "323ba850-3f40-459e-9da4-078f0eaa2730"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:57.165612 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.165585    6011 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-util\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:57.165612 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.165609    6011 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/323ba850-3f40-459e-9da4-078f0eaa2730-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:57.165775 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.165620    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v8zp\" (UniqueName: \"kubernetes.io/projected/323ba850-3f40-459e-9da4-078f0eaa2730-kube-api-access-2v8zp\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:57.832865 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.832836    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh"
Oct 30 00:45:57.833448 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.832879    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6fg8bh" event={"ID":"323ba850-3f40-459e-9da4-078f0eaa2730","Type":"ContainerDied","Data":"5b1dd857fbadfb77725c90c216e3b3e882bc2439ac8bb9a9e0dde4fb47642055"}
Oct 30 00:45:57.833448 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.832932    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b1dd857fbadfb77725c90c216e3b3e882bc2439ac8bb9a9e0dde4fb47642055"
Oct 30 00:45:57.917347 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:45:57.917302    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:45:57.983686 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:57.983658    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:58.026952 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.026932    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:45:58.030597 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.030559    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:58.075326 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075297    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-util\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-util\") pod \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") "
Oct 30 00:45:58.075508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075346    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-bundle\") pod \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") "
Oct 30 00:45:58.075508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075377    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-util\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-util\") pod \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") "
Oct 30 00:45:58.075508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075414    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-bundle\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-bundle\") pod \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") "
Oct 30 00:45:58.075508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075441    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-bundle\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-bundle\") pod \"3121e65f-0745-44fa-b848-8df1148c4421\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") "
Oct 30 00:45:58.075508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075485    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3121e65f-0745-44fa-b848-8df1148c4421-kube-api-access-kxkbk\" (UniqueName: \"kubernetes.io/projected/3121e65f-0745-44fa-b848-8df1148c4421-kube-api-access-kxkbk\") pod \"3121e65f-0745-44fa-b848-8df1148c4421\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") "
Oct 30 00:45:58.075782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075516    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-kube-api-access-4xtc6\" (UniqueName: \"kubernetes.io/projected/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-kube-api-access-4xtc6\") pod \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\" (UID: \"04663262-a0ce-4647-85cb-8e8d9e8a5bb2\") "
Oct 30 00:45:58.075782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075586    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-kube-api-access-8tmqh\" (UniqueName: \"kubernetes.io/projected/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-kube-api-access-8tmqh\") pod \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\" (UID: \"4c27010f-e26c-4c22-8ec8-d10dbf0e3084\") "
Oct 30 00:45:58.075782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075620    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-util\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-util\") pod \"3121e65f-0745-44fa-b848-8df1148c4421\" (UID: \"3121e65f-0745-44fa-b848-8df1148c4421\") "
Oct 30 00:45:58.075930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.075912    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-bundle" pod "4c27010f-e26c-4c22-8ec8-d10dbf0e3084" (UID: "4c27010f-e26c-4c22-8ec8-d10dbf0e3084"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:58.076445 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.076398    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-bundle" pod "3121e65f-0745-44fa-b848-8df1148c4421" (UID: "3121e65f-0745-44fa-b848-8df1148c4421"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:58.076951 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.076930    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-bundle" pod "04663262-a0ce-4647-85cb-8e8d9e8a5bb2" (UID: "04663262-a0ce-4647-85cb-8e8d9e8a5bb2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:58.078366 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.078332    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3121e65f-0745-44fa-b848-8df1148c4421-kube-api-access-kxkbk" pod "3121e65f-0745-44fa-b848-8df1148c4421" (UID: "3121e65f-0745-44fa-b848-8df1148c4421"). InnerVolumeSpecName "kube-api-access-kxkbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:45:58.078576 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.078549    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-kube-api-access-8tmqh" pod "4c27010f-e26c-4c22-8ec8-d10dbf0e3084" (UID: "4c27010f-e26c-4c22-8ec8-d10dbf0e3084"). InnerVolumeSpecName "kube-api-access-8tmqh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:45:58.079627 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.079599    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-kube-api-access-4xtc6" pod "04663262-a0ce-4647-85cb-8e8d9e8a5bb2" (UID: "04663262-a0ce-4647-85cb-8e8d9e8a5bb2"). InnerVolumeSpecName "kube-api-access-4xtc6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:45:58.082599 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.082574    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-util" pod "4c27010f-e26c-4c22-8ec8-d10dbf0e3084" (UID: "4c27010f-e26c-4c22-8ec8-d10dbf0e3084"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:58.083294 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.083222    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-util" pod "04663262-a0ce-4647-85cb-8e8d9e8a5bb2" (UID: "04663262-a0ce-4647-85cb-8e8d9e8a5bb2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:58.085046 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.085022    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-util" pod "3121e65f-0745-44fa-b848-8df1148c4421" (UID: "3121e65f-0745-44fa-b848-8df1148c4421"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:45:58.176718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176673    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tmqh\" (UniqueName: \"kubernetes.io/projected/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-kube-api-access-8tmqh\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176715    6011 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-util\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176732    6011 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-util\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176745    6011 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4c27010f-e26c-4c22-8ec8-d10dbf0e3084-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176757    6011 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-util\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176768    6011 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176778    6011 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3121e65f-0745-44fa-b848-8df1148c4421-bundle\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176790    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxkbk\" (UniqueName: \"kubernetes.io/projected/3121e65f-0745-44fa-b848-8df1148c4421-kube-api-access-kxkbk\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.176945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.176803    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4xtc6\" (UniqueName: \"kubernetes.io/projected/04663262-a0ce-4647-85cb-8e8d9e8a5bb2-kube-api-access-4xtc6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:45:58.839345 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.839316    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5"
Oct 30 00:45:58.839345 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.839336    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cmtpf5" event={"ID":"04663262-a0ce-4647-85cb-8e8d9e8a5bb2","Type":"ContainerDied","Data":"6394ecce01fa7e42634881699cf4131b0952b9dad601c0c58496070c8a3a4993"}
Oct 30 00:45:58.839895 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.839383    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6394ecce01fa7e42634881699cf4131b0952b9dad601c0c58496070c8a3a4993"
Oct 30 00:45:58.841332 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.841308    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc" event={"ID":"4c27010f-e26c-4c22-8ec8-d10dbf0e3084","Type":"ContainerDied","Data":"8e655e1d808fe3dc041e8cc99f438153c70d43358715e0235ca5d92418d04523"}
Oct 30 00:45:58.841478 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.841340    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8e655e1d808fe3dc041e8cc99f438153c70d43358715e0235ca5d92418d04523"
Oct 30 00:45:58.841478 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.841314    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7zsskc"
Oct 30 00:45:58.843301 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.843276    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5" event={"ID":"3121e65f-0745-44fa-b848-8df1148c4421","Type":"ContainerDied","Data":"c4657bd8c4a2bd5eada7002d77f33e88aa2441832ed45e66e87eb899698cc29f"}
Oct 30 00:45:58.843409 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.843314    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4657bd8c4a2bd5eada7002d77f33e88aa2441832ed45e66e87eb899698cc29f"
Oct 30 00:45:58.843409 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:45:58.843321    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847ax2rh5"
Oct 30 00:46:14.297142 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297101    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"]
Oct 30 00:46:14.297593 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297580    6011 cpu_manager.go:414] "containerMap: removing container" podUID="3121e65f-0745-44fa-b848-8df1148c4421" containerName="pull"
Oct 30 00:46:14.297645 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297599    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3121e65f-0745-44fa-b848-8df1148c4421" containerName="pull"
Oct 30 00:46:14.297645 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297614    6011 cpu_manager.go:414] "containerMap: removing container" podUID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerName="extract"
Oct 30 00:46:14.297645 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297619    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerName="extract"
Oct 30 00:46:14.297645 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297631    6011 cpu_manager.go:414] "containerMap: removing container" podUID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerName="util"
Oct 30 00:46:14.297645 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297638    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerName="util"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297649    6011 cpu_manager.go:414] "containerMap: removing container" podUID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerName="extract"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297656    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerName="extract"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297682    6011 cpu_manager.go:414] "containerMap: removing container" podUID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerName="util"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297689    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerName="util"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297699    6011 cpu_manager.go:414] "containerMap: removing container" podUID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerName="pull"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297707    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerName="pull"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297718    6011 cpu_manager.go:414] "containerMap: removing container" podUID="3121e65f-0745-44fa-b848-8df1148c4421" containerName="util"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297725    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3121e65f-0745-44fa-b848-8df1148c4421" containerName="util"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297747    6011 cpu_manager.go:414] "containerMap: removing container" podUID="323ba850-3f40-459e-9da4-078f0eaa2730" containerName="util"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297755    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="323ba850-3f40-459e-9da4-078f0eaa2730" containerName="util"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297773    6011 cpu_manager.go:414] "containerMap: removing container" podUID="323ba850-3f40-459e-9da4-078f0eaa2730" containerName="extract"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297779    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="323ba850-3f40-459e-9da4-078f0eaa2730" containerName="extract"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297788    6011 cpu_manager.go:414] "containerMap: removing container" podUID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerName="pull"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297794    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerName="pull"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297804    6011 cpu_manager.go:414] "containerMap: removing container" podUID="3121e65f-0745-44fa-b848-8df1148c4421" containerName="extract"
Oct 30 00:46:14.297811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297811    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3121e65f-0745-44fa-b848-8df1148c4421" containerName="extract"
Oct 30 00:46:14.298385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297825    6011 cpu_manager.go:414] "containerMap: removing container" podUID="323ba850-3f40-459e-9da4-078f0eaa2730" containerName="pull"
Oct 30 00:46:14.298385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297834    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="323ba850-3f40-459e-9da4-078f0eaa2730" containerName="pull"
Oct 30 00:46:14.298385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297912    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="3121e65f-0745-44fa-b848-8df1148c4421" containerName="extract"
Oct 30 00:46:14.298385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297924    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="323ba850-3f40-459e-9da4-078f0eaa2730" containerName="extract"
Oct 30 00:46:14.298385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297931    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c27010f-e26c-4c22-8ec8-d10dbf0e3084" containerName="extract"
Oct 30 00:46:14.298385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.297937    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="04663262-a0ce-4647-85cb-8e8d9e8a5bb2" containerName="extract"
Oct 30 00:46:14.302935 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.302917    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"
Oct 30 00:46:14.322591 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.319949    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 00:46:14.331313 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.331290    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 00:46:14.331659 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.331643    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 00:46:14.331736 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.331689    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pn7d5\""
Oct 30 00:46:14.333081 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.333065    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"]
Oct 30 00:46:14.441438 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.441394    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jldch\" (UniqueName: \"kubernetes.io/projected/869d4394-feed-4576-a544-4dc0b54c1018-kube-api-access-jldch\") pod \"dns-operator-controller-manager-65bfd79598-lsd2f\" (UID: \"869d4394-feed-4576-a544-4dc0b54c1018\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"
Oct 30 00:46:14.542148 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.542115    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jldch\" (UniqueName: \"kubernetes.io/projected/869d4394-feed-4576-a544-4dc0b54c1018-kube-api-access-jldch\") pod \"dns-operator-controller-manager-65bfd79598-lsd2f\" (UID: \"869d4394-feed-4576-a544-4dc0b54c1018\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"
Oct 30 00:46:14.561621 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.561559    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jldch\" (UniqueName: \"kubernetes.io/projected/869d4394-feed-4576-a544-4dc0b54c1018-kube-api-access-jldch\") pod \"dns-operator-controller-manager-65bfd79598-lsd2f\" (UID: \"869d4394-feed-4576-a544-4dc0b54c1018\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"
Oct 30 00:46:14.616560 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.616525    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"
Oct 30 00:46:14.911525 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:14.911501    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"]
Oct 30 00:46:14.914005 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:46:14.913976    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869d4394_feed_4576_a544_4dc0b54c1018.slice/crio-8f28e9c94e2252c92ed817a171da088898dcf40f582cca4a61c0871cdec0b343 WatchSource:0}: Error finding container 8f28e9c94e2252c92ed817a171da088898dcf40f582cca4a61c0871cdec0b343: Status 404 returned error can't find the container with id 8f28e9c94e2252c92ed817a171da088898dcf40f582cca4a61c0871cdec0b343
Oct 30 00:46:15.925687 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:15.925651    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f" event={"ID":"869d4394-feed-4576-a544-4dc0b54c1018","Type":"ContainerStarted","Data":"8f28e9c94e2252c92ed817a171da088898dcf40f582cca4a61c0871cdec0b343"}
Oct 30 00:46:18.113959 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.113923    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"]
Oct 30 00:46:18.122820 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.122790    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.136752 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.136719    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"]
Oct 30 00:46:18.147397 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.147368    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-fblw9\""
Oct 30 00:46:18.147397 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.147376    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Oct 30 00:46:18.147397 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.147392    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Oct 30 00:46:18.277286 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.277253    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d47vc\" (UniqueName: \"kubernetes.io/projected/93246f44-8c47-4f83-a435-0ec11e3d4436-kube-api-access-d47vc\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.277471 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.277313    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/93246f44-8c47-4f83-a435-0ec11e3d4436-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.277471 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.277382    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/93246f44-8c47-4f83-a435-0ec11e3d4436-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.378528 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.378453    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d47vc\" (UniqueName: \"kubernetes.io/projected/93246f44-8c47-4f83-a435-0ec11e3d4436-kube-api-access-d47vc\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.378713 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.378540    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/93246f44-8c47-4f83-a435-0ec11e3d4436-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.378713 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.378564    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/93246f44-8c47-4f83-a435-0ec11e3d4436-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.379223 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.379200    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/93246f44-8c47-4f83-a435-0ec11e3d4436-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.381147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.381128    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/93246f44-8c47-4f83-a435-0ec11e3d4436-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.398646 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.398615    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d47vc\" (UniqueName: \"kubernetes.io/projected/93246f44-8c47-4f83-a435-0ec11e3d4436-kube-api-access-d47vc\") pod \"kuadrant-console-plugin-66c4fb575d-ltv2b\" (UID: \"93246f44-8c47-4f83-a435-0ec11e3d4436\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.437620 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.437586    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"
Oct 30 00:46:18.740405 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.740317    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b"]
Oct 30 00:46:18.743711 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:46:18.743678    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93246f44_8c47_4f83_a435_0ec11e3d4436.slice/crio-56a6f8c63ede493d16014689e60f93e08bd054fc25a1704f201cf462693537fa WatchSource:0}: Error finding container 56a6f8c63ede493d16014689e60f93e08bd054fc25a1704f201cf462693537fa: Status 404 returned error can't find the container with id 56a6f8c63ede493d16014689e60f93e08bd054fc25a1704f201cf462693537fa
Oct 30 00:46:18.839986 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.839962    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Oct 30 00:46:18.944187 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.944127    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f" event={"ID":"869d4394-feed-4576-a544-4dc0b54c1018","Type":"ContainerStarted","Data":"5b1a6f717935359921c37325163c017fa8d30ce87e1b38d7f7586d3391be5f70"}
Oct 30 00:46:18.944384 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.944360    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"
Oct 30 00:46:18.945650 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.945616    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b" event={"ID":"93246f44-8c47-4f83-a435-0ec11e3d4436","Type":"ContainerStarted","Data":"56a6f8c63ede493d16014689e60f93e08bd054fc25a1704f201cf462693537fa"}
Oct 30 00:46:18.983561 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:18.983516    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f" podStartSLOduration=1.074167889 podStartE2EDuration="4.983503976s" totalImagesPullingTime="3.909336087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:18.983018996 +0000 UTC m=+421.085772994" watchObservedRunningTime="2026-10-30 00:46:18.983503976 +0000 UTC m=+421.086257946"
Oct 30 00:46:27.918118 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:46:27.918073    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:46:29.952880 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:29.952842    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-lsd2f"
Oct 30 00:46:30.756438 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:30.756345    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"]
Oct 30 00:46:30.760296 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:30.760272    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:30.776838 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:30.776808    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"]
Oct 30 00:46:30.787043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:30.786995    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cld2j\""
Oct 30 00:46:30.906274 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:30.906234    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-bzg92\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:30.906274 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:30.906273    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzqtg\" (UniqueName: \"kubernetes.io/projected/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-kube-api-access-mzqtg\") pod \"kuadrant-operator-controller-manager-6b6fb56846-bzg92\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:31.006965 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.006876    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-bzg92\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:31.006965 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.006906    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzqtg\" (UniqueName: \"kubernetes.io/projected/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-kube-api-access-mzqtg\") pod \"kuadrant-operator-controller-manager-6b6fb56846-bzg92\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:31.007552 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.007285    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b6fb56846-bzg92\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:31.009924 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.009901    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b" event={"ID":"93246f44-8c47-4f83-a435-0ec11e3d4436","Type":"ContainerStarted","Data":"17bf3a59e4939c858dd158f107a10f7471f89b71272892f25a740218ab0f15bc"}
Oct 30 00:46:31.033414 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.033383    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzqtg\" (UniqueName: \"kubernetes.io/projected/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-kube-api-access-mzqtg\") pod \"kuadrant-operator-controller-manager-6b6fb56846-bzg92\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:31.050916 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.050865    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-ltv2b" podStartSLOduration=1.413373824 podStartE2EDuration="13.050848659s" totalImagesPullingTime="11.637474835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:31.049745987 +0000 UTC m=+433.152499960" watchObservedRunningTime="2026-10-30 00:46:31.050848659 +0000 UTC m=+433.153602668"
Oct 30 00:46:31.075327 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.075284    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:31.400353 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:31.400320    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"]
Oct 30 00:46:31.402782 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:46:31.402749    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e59ffc5_6514_47e2_b059_4b13dc8bee7d.slice/crio-df70aa617f6ec5a5e1770827da5d7cea19c9d0220566180b8d053ed930777336 WatchSource:0}: Error finding container df70aa617f6ec5a5e1770827da5d7cea19c9d0220566180b8d053ed930777336: Status 404 returned error can't find the container with id df70aa617f6ec5a5e1770827da5d7cea19c9d0220566180b8d053ed930777336
Oct 30 00:46:32.019536 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:32.019500    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92" event={"ID":"7e59ffc5-6514-47e2-b059-4b13dc8bee7d","Type":"ContainerStarted","Data":"df70aa617f6ec5a5e1770827da5d7cea19c9d0220566180b8d053ed930777336"}
Oct 30 00:46:36.038012 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:36.037978    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92" event={"ID":"7e59ffc5-6514-47e2-b059-4b13dc8bee7d","Type":"ContainerStarted","Data":"c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e"}
Oct 30 00:46:36.038448 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:36.038081    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:36.103861 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:36.103808    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92" podStartSLOduration=1.9597754790000002 podStartE2EDuration="6.103791522s" totalImagesPullingTime="4.144016043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:46:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:46:36.090890894 +0000 UTC m=+438.193644900" watchObservedRunningTime="2026-10-30 00:46:36.103791522 +0000 UTC m=+438.206545510"
Oct 30 00:46:43.238735 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.238691    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"]
Oct 30 00:46:43.239224 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.238888    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92" podUID="7e59ffc5-6514-47e2-b059-4b13dc8bee7d" containerName="manager" containerID="cri-o://c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e" gracePeriod=10
Oct 30 00:46:43.240861 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.240836    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:43.493103 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.493039    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:43.521476 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.521445    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-extensions-socket-volume\") pod \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") "
Oct 30 00:46:43.521626 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.521497    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-kube-api-access-mzqtg\" (UniqueName: \"kubernetes.io/projected/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-kube-api-access-mzqtg\") pod \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\" (UID: \"7e59ffc5-6514-47e2-b059-4b13dc8bee7d\") "
Oct 30 00:46:43.521938 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.521911    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-extensions-socket-volume" pod "7e59ffc5-6514-47e2-b059-4b13dc8bee7d" (UID: "7e59ffc5-6514-47e2-b059-4b13dc8bee7d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 00:46:43.523815 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.523790    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-kube-api-access-mzqtg" pod "7e59ffc5-6514-47e2-b059-4b13dc8bee7d" (UID: "7e59ffc5-6514-47e2-b059-4b13dc8bee7d"). InnerVolumeSpecName "kube-api-access-mzqtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:46:43.623058 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.623018    6011 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-extensions-socket-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:46:43.623058 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:43.623049    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzqtg\" (UniqueName: \"kubernetes.io/projected/7e59ffc5-6514-47e2-b059-4b13dc8bee7d-kube-api-access-mzqtg\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:46:44.076949 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.076908    6011 generic.go:356] "Generic (PLEG): container finished" podID="7e59ffc5-6514-47e2-b059-4b13dc8bee7d" containerID="c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e" exitCode=0
Oct 30 00:46:44.077116 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.076962    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"
Oct 30 00:46:44.077116 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.076992    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92" event={"ID":"7e59ffc5-6514-47e2-b059-4b13dc8bee7d","Type":"ContainerDied","Data":"c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e"}
Oct 30 00:46:44.077116 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.077044    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92" event={"ID":"7e59ffc5-6514-47e2-b059-4b13dc8bee7d","Type":"ContainerDied","Data":"df70aa617f6ec5a5e1770827da5d7cea19c9d0220566180b8d053ed930777336"}
Oct 30 00:46:44.077116 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.077065    6011 scope.go:122] "RemoveContainer" containerID="c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e"
Oct 30 00:46:44.086155 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.086133    6011 scope.go:122] "RemoveContainer" containerID="c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e"
Oct 30 00:46:44.086475 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:46:44.086453    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e\": container with ID starting with c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e not found: ID does not exist" containerID="c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e"
Oct 30 00:46:44.086539 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.086488    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e"} err="failed to get container status \"c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e\": rpc error: code = NotFound desc = could not find container \"c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e\": container with ID starting with c528e2212f87e8cd97636cf0aa26df488eaf96935287cfe525ebe9060bc5371e not found: ID does not exist"
Oct 30 00:46:44.128963 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.128936    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"]
Oct 30 00:46:44.140969 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:44.140948    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b6fb56846-bzg92"]
Oct 30 00:46:46.019798 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:46:46.019762    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e59ffc5-6514-47e2-b059-4b13dc8bee7d" path="/var/lib/kubelet/pods/7e59ffc5-6514-47e2-b059-4b13dc8bee7d/volumes"
Oct 30 00:46:57.909213 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:46:57.909158    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:47:17.188954 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.188924    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5txw"]
Oct 30 00:47:17.189483 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.189322    6011 cpu_manager.go:414] "containerMap: removing container" podUID="7e59ffc5-6514-47e2-b059-4b13dc8bee7d" containerName="manager"
Oct 30 00:47:17.189483 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.189335    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e59ffc5-6514-47e2-b059-4b13dc8bee7d" containerName="manager"
Oct 30 00:47:17.189483 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.189400    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e59ffc5-6514-47e2-b059-4b13dc8bee7d" containerName="manager"
Oct 30 00:47:17.204042 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.204019    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5txw"
Oct 30 00:47:17.208424 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.208394    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5txw"]
Oct 30 00:47:17.218863 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.218826    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6b9qr\""
Oct 30 00:47:17.335153 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.335118    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtnrj\" (UniqueName: \"kubernetes.io/projected/6fbce80c-ccee-4aad-b195-f42141a1f5e7-kube-api-access-gtnrj\") pod \"authorino-57f7f8659f-x5txw\" (UID: \"6fbce80c-ccee-4aad-b195-f42141a1f5e7\") " pod="kuadrant-system/authorino-57f7f8659f-x5txw"
Oct 30 00:47:17.435896 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.435866    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtnrj\" (UniqueName: \"kubernetes.io/projected/6fbce80c-ccee-4aad-b195-f42141a1f5e7-kube-api-access-gtnrj\") pod \"authorino-57f7f8659f-x5txw\" (UID: \"6fbce80c-ccee-4aad-b195-f42141a1f5e7\") " pod="kuadrant-system/authorino-57f7f8659f-x5txw"
Oct 30 00:47:17.455042 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.454974    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtnrj\" (UniqueName: \"kubernetes.io/projected/6fbce80c-ccee-4aad-b195-f42141a1f5e7-kube-api-access-gtnrj\") pod \"authorino-57f7f8659f-x5txw\" (UID: \"6fbce80c-ccee-4aad-b195-f42141a1f5e7\") " pod="kuadrant-system/authorino-57f7f8659f-x5txw"
Oct 30 00:47:17.514984 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.514943    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5txw"
Oct 30 00:47:17.807968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.807936    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5shts"]
Oct 30 00:47:17.821345 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.821307    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5txw"]
Oct 30 00:47:17.821450 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.821418    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5shts"
Oct 30 00:47:17.826745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.826724    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5shts"]
Oct 30 00:47:17.941195 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:17.941144    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cmgq\" (UniqueName: \"kubernetes.io/projected/16d25af4-626f-431f-98c0-8616a234b157-kube-api-access-7cmgq\") pod \"authorino-56c6c9ddd5-5shts\" (UID: \"16d25af4-626f-431f-98c0-8616a234b157\") " pod="kuadrant-system/authorino-56c6c9ddd5-5shts"
Oct 30 00:47:18.042317 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:18.042290    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cmgq\" (UniqueName: \"kubernetes.io/projected/16d25af4-626f-431f-98c0-8616a234b157-kube-api-access-7cmgq\") pod \"authorino-56c6c9ddd5-5shts\" (UID: \"16d25af4-626f-431f-98c0-8616a234b157\") " pod="kuadrant-system/authorino-56c6c9ddd5-5shts"
Oct 30 00:47:18.061553 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:18.061529    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cmgq\" (UniqueName: \"kubernetes.io/projected/16d25af4-626f-431f-98c0-8616a234b157-kube-api-access-7cmgq\") pod \"authorino-56c6c9ddd5-5shts\" (UID: \"16d25af4-626f-431f-98c0-8616a234b157\") " pod="kuadrant-system/authorino-56c6c9ddd5-5shts"
Oct 30 00:47:18.134370 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:18.134344    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5shts"
Oct 30 00:47:18.248021 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:18.247987    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5txw" event={"ID":"6fbce80c-ccee-4aad-b195-f42141a1f5e7","Type":"ContainerStarted","Data":"894bd6fe3be0e8f2a17359578371d7d0f482e90879e1d9d30891454b98127d3a"}
Oct 30 00:47:18.422774 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:18.422744    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5shts"]
Oct 30 00:47:18.425079 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:47:18.425047    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16d25af4_626f_431f_98c0_8616a234b157.slice/crio-442ec5629d721d1b324fa094c1634280e215a1a4d55ada983cd61b50c8e5ac0b WatchSource:0}: Error finding container 442ec5629d721d1b324fa094c1634280e215a1a4d55ada983cd61b50c8e5ac0b: Status 404 returned error can't find the container with id 442ec5629d721d1b324fa094c1634280e215a1a4d55ada983cd61b50c8e5ac0b
Oct 30 00:47:19.255069 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:19.255033    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5shts" event={"ID":"16d25af4-626f-431f-98c0-8616a234b157","Type":"ContainerStarted","Data":"442ec5629d721d1b324fa094c1634280e215a1a4d55ada983cd61b50c8e5ac0b"}
Oct 30 00:47:20.262090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:20.262056    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5shts" event={"ID":"16d25af4-626f-431f-98c0-8616a234b157","Type":"ContainerStarted","Data":"af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6"}
Oct 30 00:47:20.299775 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:20.299705    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-56c6c9ddd5-5shts" podStartSLOduration=1.5596296170000001 podStartE2EDuration="3.299683441s" totalImagesPullingTime="1.740053824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:20.298509086 +0000 UTC m=+482.401263057" watchObservedRunningTime="2026-10-30 00:47:20.299683441 +0000 UTC m=+482.402437410"
Oct 30 00:47:20.322803 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:20.322770    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5txw"]
Oct 30 00:47:21.268906 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:21.268868    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5txw" event={"ID":"6fbce80c-ccee-4aad-b195-f42141a1f5e7","Type":"ContainerStarted","Data":"fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660"}
Oct 30 00:47:21.268906 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:21.268877    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-57f7f8659f-x5txw" podUID="6fbce80c-ccee-4aad-b195-f42141a1f5e7" containerName="authorino" containerID="cri-o://fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660" gracePeriod=30
Oct 30 00:47:21.308987 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:21.306739    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-57f7f8659f-x5txw" podStartSLOduration=1.728501536 podStartE2EDuration="4.306721904s" totalImagesPullingTime="2.578220368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:21.304544517 +0000 UTC m=+483.407298485" watchObservedRunningTime="2026-10-30 00:47:21.306721904 +0000 UTC m=+483.409475879"
Oct 30 00:47:21.516372 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:21.516349    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5txw"
Oct 30 00:47:21.678409 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:21.678373    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fbce80c-ccee-4aad-b195-f42141a1f5e7-kube-api-access-gtnrj\" (UniqueName: \"kubernetes.io/projected/6fbce80c-ccee-4aad-b195-f42141a1f5e7-kube-api-access-gtnrj\") pod \"6fbce80c-ccee-4aad-b195-f42141a1f5e7\" (UID: \"6fbce80c-ccee-4aad-b195-f42141a1f5e7\") "
Oct 30 00:47:21.680684 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:21.680655    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fbce80c-ccee-4aad-b195-f42141a1f5e7-kube-api-access-gtnrj" pod "6fbce80c-ccee-4aad-b195-f42141a1f5e7" (UID: "6fbce80c-ccee-4aad-b195-f42141a1f5e7"). InnerVolumeSpecName "kube-api-access-gtnrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:47:21.779285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:21.779244    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtnrj\" (UniqueName: \"kubernetes.io/projected/6fbce80c-ccee-4aad-b195-f42141a1f5e7-kube-api-access-gtnrj\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:47:22.274885 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.274853    6011 generic.go:356] "Generic (PLEG): container finished" podID="6fbce80c-ccee-4aad-b195-f42141a1f5e7" containerID="fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660" exitCode=0
Oct 30 00:47:22.275409 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.274898    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57f7f8659f-x5txw"
Oct 30 00:47:22.275409 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.274940    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5txw" event={"ID":"6fbce80c-ccee-4aad-b195-f42141a1f5e7","Type":"ContainerDied","Data":"fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660"}
Oct 30 00:47:22.275409 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.274988    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57f7f8659f-x5txw" event={"ID":"6fbce80c-ccee-4aad-b195-f42141a1f5e7","Type":"ContainerDied","Data":"894bd6fe3be0e8f2a17359578371d7d0f482e90879e1d9d30891454b98127d3a"}
Oct 30 00:47:22.275409 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.275010    6011 scope.go:122] "RemoveContainer" containerID="fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660"
Oct 30 00:47:22.283685 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.283666    6011 scope.go:122] "RemoveContainer" containerID="fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660"
Oct 30 00:47:22.283996 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:47:22.283973    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660\": container with ID starting with fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660 not found: ID does not exist" containerID="fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660"
Oct 30 00:47:22.284043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.284007    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660"} err="failed to get container status \"fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660\": rpc error: code = NotFound desc = could not find container \"fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660\": container with ID starting with fea460926d7e7e0d5533bf1ee9dc8a42e6c2e37b7d62e9d0c9665951eca1a660 not found: ID does not exist"
Oct 30 00:47:22.325418 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.325387    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5txw"]
Oct 30 00:47:22.337167 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:22.337135    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57f7f8659f-x5txw"]
Oct 30 00:47:24.019034 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:24.019001    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fbce80c-ccee-4aad-b195-f42141a1f5e7" path="/var/lib/kubelet/pods/6fbce80c-ccee-4aad-b195-f42141a1f5e7/volumes"
Oct 30 00:47:27.909530 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:47:27.909483    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:47:35.418881 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.418849    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"]
Oct 30 00:47:35.419383 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.419256    6011 cpu_manager.go:414] "containerMap: removing container" podUID="6fbce80c-ccee-4aad-b195-f42141a1f5e7" containerName="authorino"
Oct 30 00:47:35.419383 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.419268    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fbce80c-ccee-4aad-b195-f42141a1f5e7" containerName="authorino"
Oct 30 00:47:35.419383 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.419345    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fbce80c-ccee-4aad-b195-f42141a1f5e7" containerName="authorino"
Oct 30 00:47:35.422420 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.422403    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:35.439049 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.439015    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"]
Oct 30 00:47:35.449024 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.448999    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 00:47:35.449160 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.449036    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 00:47:35.461380 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.461355    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Oct 30 00:47:35.461562 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.461448    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xq666\""
Oct 30 00:47:35.499318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.499264    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkz26\" (UniqueName: \"kubernetes.io/projected/f6cc5223-d801-42fd-82d0-f854c7a44be4-kube-api-access-lkz26\") pod \"postgres-7cdbf6dc44-t92zl\" (UID: \"f6cc5223-d801-42fd-82d0-f854c7a44be4\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:35.499510 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.499448    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc5223-d801-42fd-82d0-f854c7a44be4-data\") pod \"postgres-7cdbf6dc44-t92zl\" (UID: \"f6cc5223-d801-42fd-82d0-f854c7a44be4\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:35.600668 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.600629    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc5223-d801-42fd-82d0-f854c7a44be4-data\") pod \"postgres-7cdbf6dc44-t92zl\" (UID: \"f6cc5223-d801-42fd-82d0-f854c7a44be4\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:35.600846 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.600679    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkz26\" (UniqueName: \"kubernetes.io/projected/f6cc5223-d801-42fd-82d0-f854c7a44be4-kube-api-access-lkz26\") pod \"postgres-7cdbf6dc44-t92zl\" (UID: \"f6cc5223-d801-42fd-82d0-f854c7a44be4\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:35.601018 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.600998    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/f6cc5223-d801-42fd-82d0-f854c7a44be4-data\") pod \"postgres-7cdbf6dc44-t92zl\" (UID: \"f6cc5223-d801-42fd-82d0-f854c7a44be4\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:35.620059 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.620030    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkz26\" (UniqueName: \"kubernetes.io/projected/f6cc5223-d801-42fd-82d0-f854c7a44be4-kube-api-access-lkz26\") pod \"postgres-7cdbf6dc44-t92zl\" (UID: \"f6cc5223-d801-42fd-82d0-f854c7a44be4\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:35.734816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:35.734727    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:36.028377 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:36.028354    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"]
Oct 30 00:47:36.030601 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:47:36.030571    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6cc5223_d801_42fd_82d0_f854c7a44be4.slice/crio-d0a1baf418b07e08925ac0e1e1186fc0b3fd95a816de409d62bc2185c72e2be0 WatchSource:0}: Error finding container d0a1baf418b07e08925ac0e1e1186fc0b3fd95a816de409d62bc2185c72e2be0: Status 404 returned error can't find the container with id d0a1baf418b07e08925ac0e1e1186fc0b3fd95a816de409d62bc2185c72e2be0
Oct 30 00:47:36.346513 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:36.346480    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl" event={"ID":"f6cc5223-d801-42fd-82d0-f854c7a44be4","Type":"ContainerStarted","Data":"d0a1baf418b07e08925ac0e1e1186fc0b3fd95a816de409d62bc2185c72e2be0"}
Oct 30 00:47:41.379206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:41.379156    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl" event={"ID":"f6cc5223-d801-42fd-82d0-f854c7a44be4","Type":"ContainerStarted","Data":"9e7f2ab470ee31d01e37802802096b6175dabbba7e8700be0c7b242da4bd3ad5"}
Oct 30 00:47:41.379644 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:41.379298    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:41.418290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:41.418240    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl" podStartSLOduration=1.643053104 podStartE2EDuration="6.418225274s" totalImagesPullingTime="4.77517217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:47:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:47:41.417122966 +0000 UTC m=+503.519876939" watchObservedRunningTime="2026-10-30 00:47:41.418225274 +0000 UTC m=+503.520979270"
Oct 30 00:47:47.415156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:47.415113    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-t92zl"
Oct 30 00:47:48.193712 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.193672    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bj4k8"]
Oct 30 00:47:48.200164 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.200136    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bj4k8"
Oct 30 00:47:48.211885 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.211861    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bj4k8"]
Oct 30 00:47:48.320196 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.320137    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqkw5\" (UniqueName: \"kubernetes.io/projected/55620684-c544-4824-9cd8-424e5ef79882-kube-api-access-cqkw5\") pod \"authorino-6f6944bcdd-bj4k8\" (UID: \"55620684-c544-4824-9cd8-424e5ef79882\") " pod="kuadrant-system/authorino-6f6944bcdd-bj4k8"
Oct 30 00:47:48.358647 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.358622    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bj4k8"]
Oct 30 00:47:48.358819 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:47:48.358803    6011 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-cqkw5], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6f6944bcdd-bj4k8" podUID="55620684-c544-4824-9cd8-424e5ef79882"
Oct 30 00:47:48.414113 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.414082    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bj4k8"
Oct 30 00:47:48.420659 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.420638    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqkw5\" (UniqueName: \"kubernetes.io/projected/55620684-c544-4824-9cd8-424e5ef79882-kube-api-access-cqkw5\") pod \"authorino-6f6944bcdd-bj4k8\" (UID: \"55620684-c544-4824-9cd8-424e5ef79882\") " pod="kuadrant-system/authorino-6f6944bcdd-bj4k8"
Oct 30 00:47:48.421396 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.421381    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bj4k8"
Oct 30 00:47:48.442346 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.442318    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqkw5\" (UniqueName: \"kubernetes.io/projected/55620684-c544-4824-9cd8-424e5ef79882-kube-api-access-cqkw5\") pod \"authorino-6f6944bcdd-bj4k8\" (UID: \"55620684-c544-4824-9cd8-424e5ef79882\") " pod="kuadrant-system/authorino-6f6944bcdd-bj4k8"
Oct 30 00:47:48.521795 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.521711    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55620684-c544-4824-9cd8-424e5ef79882-kube-api-access-cqkw5\" (UniqueName: \"kubernetes.io/projected/55620684-c544-4824-9cd8-424e5ef79882-kube-api-access-cqkw5\") pod \"55620684-c544-4824-9cd8-424e5ef79882\" (UID: \"55620684-c544-4824-9cd8-424e5ef79882\") "
Oct 30 00:47:48.524041 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.524009    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55620684-c544-4824-9cd8-424e5ef79882-kube-api-access-cqkw5" pod "55620684-c544-4824-9cd8-424e5ef79882" (UID: "55620684-c544-4824-9cd8-424e5ef79882"). InnerVolumeSpecName "kube-api-access-cqkw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:47:48.622763 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:48.622720    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cqkw5\" (UniqueName: \"kubernetes.io/projected/55620684-c544-4824-9cd8-424e5ef79882-kube-api-access-cqkw5\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:47:49.418679 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:49.418649    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f6944bcdd-bj4k8"
Oct 30 00:47:49.493505 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:49.493473    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bj4k8"]
Oct 30 00:47:49.496601 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:49.496577    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f6944bcdd-bj4k8"]
Oct 30 00:47:50.020135 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:50.020100    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55620684-c544-4824-9cd8-424e5ef79882" path="/var/lib/kubelet/pods/55620684-c544-4824-9cd8-424e5ef79882/volumes"
Oct 30 00:47:51.524811 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:51.524772    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5shts"]
Oct 30 00:47:51.525356 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:51.524954    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/authorino-56c6c9ddd5-5shts" podUID="16d25af4-626f-431f-98c0-8616a234b157" containerName="authorino" containerID="cri-o://af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6" gracePeriod=30
Oct 30 00:47:51.772683 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:51.772658    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5shts"
Oct 30 00:47:51.853034 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:51.853002    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/16d25af4-626f-431f-98c0-8616a234b157-kube-api-access-7cmgq\" (UniqueName: \"kubernetes.io/projected/16d25af4-626f-431f-98c0-8616a234b157-kube-api-access-7cmgq\") pod \"16d25af4-626f-431f-98c0-8616a234b157\" (UID: \"16d25af4-626f-431f-98c0-8616a234b157\") "
Oct 30 00:47:51.855250 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:51.855222    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d25af4-626f-431f-98c0-8616a234b157-kube-api-access-7cmgq" pod "16d25af4-626f-431f-98c0-8616a234b157" (UID: "16d25af4-626f-431f-98c0-8616a234b157"). InnerVolumeSpecName "kube-api-access-7cmgq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:47:51.954226 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:51.954157    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cmgq\" (UniqueName: \"kubernetes.io/projected/16d25af4-626f-431f-98c0-8616a234b157-kube-api-access-7cmgq\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:47:52.435592 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.435558    6011 generic.go:356] "Generic (PLEG): container finished" podID="16d25af4-626f-431f-98c0-8616a234b157" containerID="af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6" exitCode=0
Oct 30 00:47:52.435787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.435612    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56c6c9ddd5-5shts"
Oct 30 00:47:52.435787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.435633    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5shts" event={"ID":"16d25af4-626f-431f-98c0-8616a234b157","Type":"ContainerDied","Data":"af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6"}
Oct 30 00:47:52.435787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.435680    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56c6c9ddd5-5shts" event={"ID":"16d25af4-626f-431f-98c0-8616a234b157","Type":"ContainerDied","Data":"442ec5629d721d1b324fa094c1634280e215a1a4d55ada983cd61b50c8e5ac0b"}
Oct 30 00:47:52.435787 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.435698    6011 scope.go:122] "RemoveContainer" containerID="af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6"
Oct 30 00:47:52.445765 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.445742    6011 scope.go:122] "RemoveContainer" containerID="af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6"
Oct 30 00:47:52.446096 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:47:52.446078    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6\": container with ID starting with af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6 not found: ID does not exist" containerID="af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6"
Oct 30 00:47:52.446144 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.446107    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6"} err="failed to get container status \"af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6\": rpc error: code = NotFound desc = could not find container \"af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6\": container with ID starting with af4fde238946660565dd4f03e376a9c074f943cc9fdebabe0bfdc1e26da6a7a6 not found: ID does not exist"
Oct 30 00:47:52.487347 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.487305    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5shts"]
Oct 30 00:47:52.499199 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:52.499129    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56c6c9ddd5-5shts"]
Oct 30 00:47:54.019847 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:47:54.019814    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="16d25af4-626f-431f-98c0-8616a234b157" path="/var/lib/kubelet/pods/16d25af4-626f-431f-98c0-8616a234b157/volumes"
Oct 30 00:47:57.909969 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:47:57.909927    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:48:27.909903 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:48:27.909809    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:48:36.067548 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.067507    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"]
Oct 30 00:48:36.068272 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.068245    6011 cpu_manager.go:414] "containerMap: removing container" podUID="16d25af4-626f-431f-98c0-8616a234b157" containerName="authorino"
Oct 30 00:48:36.068393 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.068272    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="16d25af4-626f-431f-98c0-8616a234b157" containerName="authorino"
Oct 30 00:48:36.068393 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.068385    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="16d25af4-626f-431f-98c0-8616a234b157" containerName="authorino"
Oct 30 00:48:36.071719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.071693    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.092151 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.092129    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"]
Oct 30 00:48:36.102143 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.102121    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6w4xf\""
Oct 30 00:48:36.102285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.102123    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 00:48:36.102285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.102124    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 00:48:36.146220 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.146161    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92549\" (UniqueName: \"kubernetes.io/projected/70d5a52c-2627-4e9c-9229-01a752fc0827-kube-api-access-92549\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.146428 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.146226    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-metrics-certs\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.146428 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.146385    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70d5a52c-2627-4e9c-9229-01a752fc0827-openshift-service-ca\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.146558 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.146523    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-maas-api-tls\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.247509 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.247463    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-metrics-certs\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.247725 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.247528    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70d5a52c-2627-4e9c-9229-01a752fc0827-openshift-service-ca\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.247725 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.247624    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-maas-api-tls\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.247725 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.247671    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92549\" (UniqueName: \"kubernetes.io/projected/70d5a52c-2627-4e9c-9229-01a752fc0827-kube-api-access-92549\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.248596 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.248568    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70d5a52c-2627-4e9c-9229-01a752fc0827-openshift-service-ca\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.250224 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.250190    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-maas-api-tls\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.250311 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.250290    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-metrics-certs\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.266819 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.266788    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-92549\" (UniqueName: \"kubernetes.io/projected/70d5a52c-2627-4e9c-9229-01a752fc0827-kube-api-access-92549\") pod \"maas-api-5c48877c75-6h4cd\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.385081 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.384994    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:36.683279 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.683200    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"]
Oct 30 00:48:36.686313 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:48:36.686283    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70d5a52c_2627_4e9c_9229_01a752fc0827.slice/crio-5b443c5baed29248244dc9d47594384fc5a11a2c0023f650c0f45b5c4bb6d2bb WatchSource:0}: Error finding container 5b443c5baed29248244dc9d47594384fc5a11a2c0023f650c0f45b5c4bb6d2bb: Status 404 returned error can't find the container with id 5b443c5baed29248244dc9d47594384fc5a11a2c0023f650c0f45b5c4bb6d2bb
Oct 30 00:48:36.687585 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:36.687568    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:48:37.672937 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:37.672896    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd" event={"ID":"70d5a52c-2627-4e9c-9229-01a752fc0827","Type":"ContainerStarted","Data":"5b443c5baed29248244dc9d47594384fc5a11a2c0023f650c0f45b5c4bb6d2bb"}
Oct 30 00:48:39.685218 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:39.685157    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd" event={"ID":"70d5a52c-2627-4e9c-9229-01a752fc0827","Type":"ContainerStarted","Data":"5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df"}
Oct 30 00:48:39.685647 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:39.685348    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:39.723757 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:39.723703    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd" podStartSLOduration=1.390738471 podStartE2EDuration="3.723689163s" totalImagesPullingTime="2.332950692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:48:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:48:39.722594101 +0000 UTC m=+561.825348072" watchObservedRunningTime="2026-10-30 00:48:39.723689163 +0000 UTC m=+561.826443134"
Oct 30 00:48:45.695919 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:48:45.695887    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 00:48:57.908913 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:48:57.908866    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:49:27.908103 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:49:27.908064    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:49:40.129528 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.129487    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"]
Oct 30 00:49:40.132950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.132927    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.162614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.162587    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 00:49:40.162614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.162610    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 00:49:40.162813 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.162622    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8fnkv\""
Oct 30 00:49:40.162813 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.162677    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 00:49:40.164372 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.164347    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"]
Oct 30 00:49:40.322479 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.322443    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.322652 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.322489    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.322652 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.322556    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.322652 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.322596    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr2tp\" (UniqueName: \"kubernetes.io/projected/ffc56f52-428f-4e01-bf96-9a42df58b065-kube-api-access-wr2tp\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.322652 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.322621    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffc56f52-428f-4e01-bf96-9a42df58b065-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.322788 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.322683    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423190 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423089    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr2tp\" (UniqueName: \"kubernetes.io/projected/ffc56f52-428f-4e01-bf96-9a42df58b065-kube-api-access-wr2tp\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423190 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423129    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffc56f52-428f-4e01-bf96-9a42df58b065-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423190 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423163    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423449 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423233    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423449 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423256    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423449 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423286    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423759 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423732    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423886 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423768    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.423886 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.423836    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.425763 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.425739    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ffc56f52-428f-4e01-bf96-9a42df58b065-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.426265 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.426244    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ffc56f52-428f-4e01-bf96-9a42df58b065-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.443321 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.443288    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr2tp\" (UniqueName: \"kubernetes.io/projected/ffc56f52-428f-4e01-bf96-9a42df58b065-kube-api-access-wr2tp\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77\" (UID: \"ffc56f52-428f-4e01-bf96-9a42df58b065\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.445460 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.445435    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:40.741602 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.741576    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"]
Oct 30 00:49:40.743655 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:49:40.743622    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podffc56f52_428f_4e01_bf96_9a42df58b065.slice/crio-c8dc31f2fd322d44f839cc6a6259541aa5a57968ccb4e7246f4fa6cbc4a98bcd WatchSource:0}: Error finding container c8dc31f2fd322d44f839cc6a6259541aa5a57968ccb4e7246f4fa6cbc4a98bcd: Status 404 returned error can't find the container with id c8dc31f2fd322d44f839cc6a6259541aa5a57968ccb4e7246f4fa6cbc4a98bcd
Oct 30 00:49:40.994446 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:40.994363    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" event={"ID":"ffc56f52-428f-4e01-bf96-9a42df58b065","Type":"ContainerStarted","Data":"c8dc31f2fd322d44f839cc6a6259541aa5a57968ccb4e7246f4fa6cbc4a98bcd"}
Oct 30 00:49:42.000983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:42.000946    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" event={"ID":"ffc56f52-428f-4e01-bf96-9a42df58b065","Type":"ContainerStarted","Data":"6833c8d6a9f7d0ea0f73a645123512e7e7c2dd9a5ceec1e5866df8f4313c5fb7"}
Oct 30 00:49:42.042085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:42.042024    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podStartSLOduration=0.95718938 podStartE2EDuration="2.042010329s" totalImagesPullingTime="1.084820949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:49:42.040301035 +0000 UTC m=+624.143055008" watchObservedRunningTime="2026-10-30 00:49:42.042010329 +0000 UTC m=+624.144764291"
Oct 30 00:49:42.445680 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:42.445633    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:45.534564 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.534531    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"]
Oct 30 00:49:45.538034 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.538012    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.556960 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.556936    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Oct 30 00:49:45.570521 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.570489    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.570521 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.570529    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxdq5\" (UniqueName: \"kubernetes.io/projected/c65bb226-5405-477b-9730-3d33e151004c-kube-api-access-vxdq5\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.570732 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.570613    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"]
Oct 30 00:49:45.570732 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.570655    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c65bb226-5405-477b-9730-3d33e151004c-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.570732 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.570693    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.570732 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.570712    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.570943 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.570828    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.671745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.671709    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c65bb226-5405-477b-9730-3d33e151004c-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.671745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.671747    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.671971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.671765    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.671971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.671795    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.671971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.671856    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.671971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.671882    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxdq5\" (UniqueName: \"kubernetes.io/projected/c65bb226-5405-477b-9730-3d33e151004c-kube-api-access-vxdq5\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.672251 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.672233    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-home\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.672319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.672296    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-model-cache\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.672388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.672324    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.674306 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.674284    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c65bb226-5405-477b-9730-3d33e151004c-dshm\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.674438 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.674423    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c65bb226-5405-477b-9730-3d33e151004c-tls-certs\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.691476 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.691449    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxdq5\" (UniqueName: \"kubernetes.io/projected/c65bb226-5405-477b-9730-3d33e151004c-kube-api-access-vxdq5\") pod \"e2e-embedding-simulated-kserve-67bf594cd6-5bhqz\" (UID: \"c65bb226-5405-477b-9730-3d33e151004c\") " pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:45.852767 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:45.852677    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:46.148793 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.148767    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"]
Oct 30 00:49:46.150957 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:49:46.150930    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65bb226_5405_477b_9730_3d33e151004c.slice/crio-30d2ddba5e5864af773f8aabaeb40ff25b44dcc8ee1291355e2cf08478a0e277 WatchSource:0}: Error finding container 30d2ddba5e5864af773f8aabaeb40ff25b44dcc8ee1291355e2cf08478a0e277: Status 404 returned error can't find the container with id 30d2ddba5e5864af773f8aabaeb40ff25b44dcc8ee1291355e2cf08478a0e277
Oct 30 00:49:46.695732 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.695696    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"]
Oct 30 00:49:46.699590 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.699562    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.718578 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.718493    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"]
Oct 30 00:49:46.728404 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.728377    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 00:49:46.782978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.782946    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.783143 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.783021    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.783143 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.783067    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d2fef78-7a92-4353-8c09-33b082f9d7b8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.783259 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.783161    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.783259 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.783216    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7wf9\" (UniqueName: \"kubernetes.io/projected/2d2fef78-7a92-4353-8c09-33b082f9d7b8-kube-api-access-p7wf9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.783342 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.783257    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.884368 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884335    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.884368 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884369    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7wf9\" (UniqueName: \"kubernetes.io/projected/2d2fef78-7a92-4353-8c09-33b082f9d7b8-kube-api-access-p7wf9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.884609 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884398    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.884609 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884435    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.884609 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884516    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.884609 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884542    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d2fef78-7a92-4353-8c09-33b082f9d7b8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.884900 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884868    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.885039 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.884903    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.885109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.885033    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.886973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.886952    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d2fef78-7a92-4353-8c09-33b082f9d7b8-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.887264 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.887247    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d2fef78-7a92-4353-8c09-33b082f9d7b8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:46.903767 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:46.903744    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7wf9\" (UniqueName: \"kubernetes.io/projected/2d2fef78-7a92-4353-8c09-33b082f9d7b8-kube-api-access-p7wf9\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-sfwrr\" (UID: \"2d2fef78-7a92-4353-8c09-33b082f9d7b8\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:47.015499 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:47.015418    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:47.027534 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:47.027504    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz" event={"ID":"c65bb226-5405-477b-9730-3d33e151004c","Type":"ContainerStarted","Data":"9d0ed06d545b677ee64571a1dcec42a841f591112d64c403afab73609c821630"}
Oct 30 00:49:47.027655 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:47.027552    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz" event={"ID":"c65bb226-5405-477b-9730-3d33e151004c","Type":"ContainerStarted","Data":"30d2ddba5e5864af773f8aabaeb40ff25b44dcc8ee1291355e2cf08478a0e277"}
Oct 30 00:49:47.072974 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:47.072908    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz" podStartSLOduration=1.8959786159999998 podStartE2EDuration="2.072888217s" totalImagesPullingTime="176.909601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:49:47.070820383 +0000 UTC m=+629.173574373" watchObservedRunningTime="2026-10-30 00:49:47.072888217 +0000 UTC m=+629.175642189"
Oct 30 00:49:47.315453 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:47.315424    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"]
Oct 30 00:49:47.317874 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:49:47.317843    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d2fef78_7a92_4353_8c09_33b082f9d7b8.slice/crio-37f74fb7e6cacbe9573edce0ab657418540ef7d3e1d1ec444169bb0004941a34 WatchSource:0}: Error finding container 37f74fb7e6cacbe9573edce0ab657418540ef7d3e1d1ec444169bb0004941a34: Status 404 returned error can't find the container with id 37f74fb7e6cacbe9573edce0ab657418540ef7d3e1d1ec444169bb0004941a34
Oct 30 00:49:47.853364 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:47.853330    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:48.033834 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:48.033801    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr" event={"ID":"2d2fef78-7a92-4353-8c09-33b082f9d7b8","Type":"ContainerStarted","Data":"53718abe35a35785eae337f0a93c850ff277bbfc5bacb0a92721f89233370365"}
Oct 30 00:49:48.034005 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:48.033852    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr" event={"ID":"2d2fef78-7a92-4353-8c09-33b082f9d7b8","Type":"ContainerStarted","Data":"37f74fb7e6cacbe9573edce0ab657418540ef7d3e1d1ec444169bb0004941a34"}
Oct 30 00:49:48.074461 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:48.074404    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr" podStartSLOduration=1.891924696 podStartE2EDuration="2.074391661s" totalImagesPullingTime="182.466965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:49:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:49:48.073158048 +0000 UTC m=+630.175912019" watchObservedRunningTime="2026-10-30 00:49:48.074391661 +0000 UTC m=+630.177145632"
Oct 30 00:49:49.015802 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:49.015768    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:50.446156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:50.446121    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:50.458766 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:50.458736    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:51.060640 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:51.060607    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:49:55.853458 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:55.853418    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:55.866339 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:55.866309    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:56.087479 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:56.087449    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-67bf594cd6-5bhqz"
Oct 30 00:49:57.016420 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:57.016376    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:57.029135 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:57.029104    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:57.095840 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:49:57.095808    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-sfwrr"
Oct 30 00:49:57.910020 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:49:57.909983    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:50:08.136268 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.136232    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-794dc9cdb5-f6dgm"]
Oct 30 00:50:08.139774 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.139752    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.169114 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.169082    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6b9qr\""
Oct 30 00:50:08.169290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.169266    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Oct 30 00:50:08.171012 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.170988    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-794dc9cdb5-f6dgm"]
Oct 30 00:50:08.289390 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.289351    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2949671-a478-4bd4-9da8-d02e13ad22f2-tls-cert\") pod \"authorino-794dc9cdb5-f6dgm\" (UID: \"d2949671-a478-4bd4-9da8-d02e13ad22f2\") " pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.289574 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.289413    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87sx6\" (UniqueName: \"kubernetes.io/projected/d2949671-a478-4bd4-9da8-d02e13ad22f2-kube-api-access-87sx6\") pod \"authorino-794dc9cdb5-f6dgm\" (UID: \"d2949671-a478-4bd4-9da8-d02e13ad22f2\") " pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.390280 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.390149    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87sx6\" (UniqueName: \"kubernetes.io/projected/d2949671-a478-4bd4-9da8-d02e13ad22f2-kube-api-access-87sx6\") pod \"authorino-794dc9cdb5-f6dgm\" (UID: \"d2949671-a478-4bd4-9da8-d02e13ad22f2\") " pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.390430 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.390312    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2949671-a478-4bd4-9da8-d02e13ad22f2-tls-cert\") pod \"authorino-794dc9cdb5-f6dgm\" (UID: \"d2949671-a478-4bd4-9da8-d02e13ad22f2\") " pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.393027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.393001    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2949671-a478-4bd4-9da8-d02e13ad22f2-tls-cert\") pod \"authorino-794dc9cdb5-f6dgm\" (UID: \"d2949671-a478-4bd4-9da8-d02e13ad22f2\") " pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.411874 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.411839    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87sx6\" (UniqueName: \"kubernetes.io/projected/d2949671-a478-4bd4-9da8-d02e13ad22f2-kube-api-access-87sx6\") pod \"authorino-794dc9cdb5-f6dgm\" (UID: \"d2949671-a478-4bd4-9da8-d02e13ad22f2\") " pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.451946 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.451912    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-794dc9cdb5-f6dgm"
Oct 30 00:50:08.750342 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:08.750316    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-794dc9cdb5-f6dgm"]
Oct 30 00:50:08.751064 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:50:08.751035    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2949671_a478_4bd4_9da8_d02e13ad22f2.slice/crio-2fa4b1efaf29f12c6bda51154ae252ee410151ae70ba0fb72f1193c1fe42850a WatchSource:0}: Error finding container 2fa4b1efaf29f12c6bda51154ae252ee410151ae70ba0fb72f1193c1fe42850a: Status 404 returned error can't find the container with id 2fa4b1efaf29f12c6bda51154ae252ee410151ae70ba0fb72f1193c1fe42850a
Oct 30 00:50:09.149000 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:09.148970    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-794dc9cdb5-f6dgm" event={"ID":"d2949671-a478-4bd4-9da8-d02e13ad22f2","Type":"ContainerStarted","Data":"2fa4b1efaf29f12c6bda51154ae252ee410151ae70ba0fb72f1193c1fe42850a"}
Oct 30 00:50:10.156223 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:10.156159    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-794dc9cdb5-f6dgm" event={"ID":"d2949671-a478-4bd4-9da8-d02e13ad22f2","Type":"ContainerStarted","Data":"b995d2fe4c548b462c27394bbb3e4c7a3aed32e4f8adca341f670aa23cf7a8fd"}
Oct 30 00:50:10.199992 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:10.199940    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-794dc9cdb5-f6dgm" podStartSLOduration=1.857057333 podStartE2EDuration="2.199921947s" totalImagesPullingTime="342.864614ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:10.198005529 +0000 UTC m=+652.300759499" watchObservedRunningTime="2026-10-30 00:50:10.199921947 +0000 UTC m=+652.302675920"
Oct 30 00:50:16.146640 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.146608    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"]
Oct 30 00:50:16.152993 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.152970    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.170471 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.170441    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 00:50:16.170471 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.170446    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rxg2r\""
Oct 30 00:50:16.185943 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.185918    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"]
Oct 30 00:50:16.271809 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.271775    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.271994 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.271829    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.271994 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.271948    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.272066 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.272001    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.272099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.272055    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.272099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.272081    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.272164 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.272140    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4pl5\" (UniqueName: \"kubernetes.io/projected/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-kube-api-access-q4pl5\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.272230 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.272202    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.272230 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.272222    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373129 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373089    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373197    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373247    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373273    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373295    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373321    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4pl5\" (UniqueName: \"kubernetes.io/projected/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-kube-api-access-q4pl5\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373586 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373391    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373586 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373483    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373586 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373559    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373738 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373586    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373738 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373696    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373887 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373866    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.373960 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.373938    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.374073 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.374058    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.375677 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.375654    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.376124 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.376108    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.397969 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.397897    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.397969 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.397915    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4pl5\" (UniqueName: \"kubernetes.io/projected/a5403c3f-cc0a-434b-9f83-080fe7a76bdb-kube-api-access-q4pl5\") pod \"maas-default-gateway-openshift-default-6db586dd6c-xw7jh\" (UID: \"a5403c3f-cc0a-434b-9f83-080fe7a76bdb\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.466599 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.466565    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:16.779449 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:16.779423    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"]
Oct 30 00:50:16.780892 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:50:16.780861    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5403c3f_cc0a_434b_9f83_080fe7a76bdb.slice/crio-bfdfe690d4765ae58ef41e39b4542a4a2fbe81a3862923085c5ef37ca988a606 WatchSource:0}: Error finding container bfdfe690d4765ae58ef41e39b4542a4a2fbe81a3862923085c5ef37ca988a606: Status 404 returned error can't find the container with id bfdfe690d4765ae58ef41e39b4542a4a2fbe81a3862923085c5ef37ca988a606
Oct 30 00:50:17.196377 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:17.196336    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" event={"ID":"a5403c3f-cc0a-434b-9f83-080fe7a76bdb","Type":"ContainerStarted","Data":"bfdfe690d4765ae58ef41e39b4542a4a2fbe81a3862923085c5ef37ca988a606"}
Oct 30 00:50:20.949823 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:20.949789    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:50:20.950094 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:20.949860    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:50:20.950094 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:20.949887    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:50:21.220406 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:21.220308    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" event={"ID":"a5403c3f-cc0a-434b-9f83-080fe7a76bdb","Type":"ContainerStarted","Data":"88429fc36d85c57e16e8635a55761f747786a0e0930712b8c4265322e3b77d22"}
Oct 30 00:50:21.297410 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:21.297360    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podStartSLOduration=1.131244183 podStartE2EDuration="5.297345698s" totalImagesPullingTime="4.166101515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:21.28348938 +0000 UTC m=+663.386243440" watchObservedRunningTime="2026-10-30 00:50:21.297345698 +0000 UTC m=+663.400099668"
Oct 30 00:50:21.467511 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:21.467468    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:22.468515 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:22.468471    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-xw7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 00:50:22.469013 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:22.468557    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podUID="a5403c3f-cc0a-434b-9f83-080fe7a76bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 00:50:23.467847 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:23.467808    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-xw7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 00:50:23.468024 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:23.467871    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podUID="a5403c3f-cc0a-434b-9f83-080fe7a76bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 00:50:24.467766 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:24.467729    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-xw7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 00:50:24.468166 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:24.467791    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podUID="a5403c3f-cc0a-434b-9f83-080fe7a76bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 00:50:25.467648 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:25.467606    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-xw7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 00:50:25.467858 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:25.467657    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podUID="a5403c3f-cc0a-434b-9f83-080fe7a76bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 00:50:26.467804 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:26.467756    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-xw7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 00:50:26.468010 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:26.467825    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podUID="a5403c3f-cc0a-434b-9f83-080fe7a76bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 00:50:27.467627 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:27.467588    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-xw7jh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Oct 30 00:50:27.467810 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:27.467644    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podUID="a5403c3f-cc0a-434b-9f83-080fe7a76bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Oct 30 00:50:27.909602 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:50:27.909564    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:50:29.226057 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:29.226005    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:29.226594 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:29.226399    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:29.227252 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:29.227234    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh"
Oct 30 00:50:31.196030 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.195996    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"]
Oct 30 00:50:31.199809 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.199788    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.234869 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.234837    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"]
Oct 30 00:50:31.246273 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246240    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246423 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246281    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d68c4f9-2522-49ac-8570-c45a39dcce25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246475 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246446    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246531 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246514    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsd9h\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-kube-api-access-xsd9h\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246580 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246565    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246621    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246675 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246647    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246724 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246705    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.246779 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.246764    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348213    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348256    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348274    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348294    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348354    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348384    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d68c4f9-2522-49ac-8570-c45a39dcce25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348443    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348725 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348496    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsd9h\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-kube-api-access-xsd9h\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348725 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348539    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348833 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348719    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348883 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348842    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348936 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348881    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.348974 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.348950    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.349205 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.349160    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d68c4f9-2522-49ac-8570-c45a39dcce25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.350960 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.350937    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.351095 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.351078    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.371465 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.371434    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.371596 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.371568    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsd9h\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-kube-api-access-xsd9h\") pod \"maas-default-gateway-openshift-default-6db586dd6c-cxjtp\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.513428 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.513345    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:31.827208 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.827085    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"]
Oct 30 00:50:31.834156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.834118    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:50:31.834269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.834209    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:50:31.834269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:31.834263    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:50:32.305311 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:32.305276    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" event={"ID":"3d68c4f9-2522-49ac-8570-c45a39dcce25","Type":"ContainerStarted","Data":"5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b"}
Oct 30 00:50:32.305715 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:32.305328    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" event={"ID":"3d68c4f9-2522-49ac-8570-c45a39dcce25","Type":"ContainerStarted","Data":"36722e1ee6102bc6b3eec279f2558c40a2113ff5f8ecb6411a38954278c0e604"}
Oct 30 00:50:32.381220 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:32.381136    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podStartSLOduration=1.381119288 podStartE2EDuration="1.381119288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:50:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:50:32.366652702 +0000 UTC m=+674.469406690" watchObservedRunningTime="2026-10-30 00:50:32.381119288 +0000 UTC m=+674.483873253"
Oct 30 00:50:32.514486 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:32.514448    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:32.516043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:32.516017    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 00:50:32.516167 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:32.516067    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 00:50:33.513786 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:33.513748    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 00:50:33.514303 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:33.513816    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 00:50:34.514593 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:34.514561    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 00:50:34.515002 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:34.514613    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 00:50:35.514231 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:35.514164    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 00:50:35.514429 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:35.514254    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 00:50:36.514509 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:36.514474    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 00:50:36.514898 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:36.514525    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 00:50:37.514574 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:37.514536    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 00:50:37.514978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:37.514593    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 00:50:38.514307 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:38.514263    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 00:50:38.514522 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:38.514354    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 00:50:40.193607 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:40.193568    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:40.194112 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:40.193901    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:40.194739 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:50:40.194719    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 00:50:57.910494 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:50:57.910453    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:51:27.909913 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:51:27.909823    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:51:36.001803 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.001766    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"]
Oct 30 00:51:36.005551 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.005533    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.031017 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.030988    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"]
Oct 30 00:51:36.040526 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.040499    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-8cf33a-dockercfg-bd2hw\""
Oct 30 00:51:36.043594 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.043574    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-8cf33a\""
Oct 30 00:51:36.096095 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.096065    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"]
Oct 30 00:51:36.105660 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.105371    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.106298 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.106269    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgcw7\" (UniqueName: \"kubernetes.io/projected/054a85f3-147d-469f-a4de-a5a29397d439-kube-api-access-dgcw7\") pod \"payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.106638 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.106614    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/054a85f3-147d-469f-a4de-a5a29397d439-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.122583 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.122554    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"]
Oct 30 00:51:36.207220 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.207147    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxb28\" (UniqueName: \"kubernetes.io/projected/863b2194-53c5-46ba-8f64-6215d4df058b-kube-api-access-gxb28\") pod \"payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.207400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.207290    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/054a85f3-147d-469f-a4de-a5a29397d439-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.207400 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.207345    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863b2194-53c5-46ba-8f64-6215d4df058b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.207474 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.207404    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgcw7\" (UniqueName: \"kubernetes.io/projected/054a85f3-147d-469f-a4de-a5a29397d439-kube-api-access-dgcw7\") pod \"payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.208062 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.208041    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/054a85f3-147d-469f-a4de-a5a29397d439-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.228541 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.228507    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgcw7\" (UniqueName: \"kubernetes.io/projected/054a85f3-147d-469f-a4de-a5a29397d439-kube-api-access-dgcw7\") pod \"payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.308632 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.308598    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863b2194-53c5-46ba-8f64-6215d4df058b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.308799 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.308703    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxb28\" (UniqueName: \"kubernetes.io/projected/863b2194-53c5-46ba-8f64-6215d4df058b-kube-api-access-gxb28\") pod \"payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.309374 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.309355    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863b2194-53c5-46ba-8f64-6215d4df058b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.318565 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.318546    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:36.333611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.333584    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxb28\" (UniqueName: \"kubernetes.io/projected/863b2194-53c5-46ba-8f64-6215d4df058b-kube-api-access-gxb28\") pod \"payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") " pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.419757 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.419727    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:36.620656 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.620629    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"]
Oct 30 00:51:36.621318 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:36.621297    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod054a85f3_147d_469f_a4de_a5a29397d439.slice/crio-462a0ba461cb5893c4bcfd67d8dace707053ff84e90bbd24c869b71479ec3c9a WatchSource:0}: Error finding container 462a0ba461cb5893c4bcfd67d8dace707053ff84e90bbd24c869b71479ec3c9a: Status 404 returned error can't find the container with id 462a0ba461cb5893c4bcfd67d8dace707053ff84e90bbd24c869b71479ec3c9a
Oct 30 00:51:36.652342 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.652304    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" event={"ID":"054a85f3-147d-469f-a4de-a5a29397d439","Type":"ContainerStarted","Data":"462a0ba461cb5893c4bcfd67d8dace707053ff84e90bbd24c869b71479ec3c9a"}
Oct 30 00:51:36.719489 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:36.719458    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"]
Oct 30 00:51:36.720226 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:36.720197    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863b2194_53c5_46ba_8f64_6215d4df058b.slice/crio-2a500f4354b25fd412bea6564e9b75fb45d2e5ce93022109942dfd04a52ea085 WatchSource:0}: Error finding container 2a500f4354b25fd412bea6564e9b75fb45d2e5ce93022109942dfd04a52ea085: Status 404 returned error can't find the container with id 2a500f4354b25fd412bea6564e9b75fb45d2e5ce93022109942dfd04a52ea085
Oct 30 00:51:37.665119 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.665075    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" event={"ID":"863b2194-53c5-46ba-8f64-6215d4df058b","Type":"ContainerStarted","Data":"2a500f4354b25fd412bea6564e9b75fb45d2e5ce93022109942dfd04a52ea085"}
Oct 30 00:51:37.869753 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.869717    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"]
Oct 30 00:51:37.874137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.874112    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:37.893443 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.893411    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-7d2b75\""
Oct 30 00:51:37.894986 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.894963    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"]
Oct 30 00:51:37.906399 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.906371    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-7d2b75-dockercfg-fpvzf\""
Oct 30 00:51:37.958460 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.958375    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"]
Oct 30 00:51:37.962871 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.962837    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:37.981703 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:37.981635    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"]
Oct 30 00:51:38.027520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.027492    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrjzm\" (UniqueName: \"kubernetes.io/projected/df652682-5a54-4e2e-afdb-25d2dffe9953-kube-api-access-xrjzm\") pod \"payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:38.027700 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.027561    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df652682-5a54-4e2e-afdb-25d2dffe9953-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:38.129609 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.129568    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrjzm\" (UniqueName: \"kubernetes.io/projected/df652682-5a54-4e2e-afdb-25d2dffe9953-kube-api-access-xrjzm\") pod \"payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:38.129824 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.129620    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:38.129824 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.129700    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns752\" (UniqueName: \"kubernetes.io/projected/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-kube-api-access-ns752\") pod \"payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:38.129824 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.129751    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df652682-5a54-4e2e-afdb-25d2dffe9953-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:38.130728 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.130698    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df652682-5a54-4e2e-afdb-25d2dffe9953-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:38.152332 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.152293    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrjzm\" (UniqueName: \"kubernetes.io/projected/df652682-5a54-4e2e-afdb-25d2dffe9953-kube-api-access-xrjzm\") pod \"payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:38.191678 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.191644    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:38.231743 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.231120    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:38.231743 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.231220    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns752\" (UniqueName: \"kubernetes.io/projected/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-kube-api-access-ns752\") pod \"payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:38.232002 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.231957    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-plugins-config-volume\") pod \"payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:38.252682 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.252646    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns752\" (UniqueName: \"kubernetes.io/projected/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-kube-api-access-ns752\") pod \"payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") " pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:38.283368 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.282856    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:38.515053 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.514981    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"]
Oct 30 00:51:38.518864 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:38.518830    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf652682_5a54_4e2e_afdb_25d2dffe9953.slice/crio-7d6a12a4cc5fd3376cb19898159af3df563ce59ed56cdf9188178006c057395e WatchSource:0}: Error finding container 7d6a12a4cc5fd3376cb19898159af3df563ce59ed56cdf9188178006c057395e: Status 404 returned error can't find the container with id 7d6a12a4cc5fd3376cb19898159af3df563ce59ed56cdf9188178006c057395e
Oct 30 00:51:38.535830 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.535799    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"]
Oct 30 00:51:38.541548 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.541528    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.561483 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.561451    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"]
Oct 30 00:51:38.570732 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.570707    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-8cf33a-serving-cert\""
Oct 30 00:51:38.601746 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.601713    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"]
Oct 30 00:51:38.636117 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.636072    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.636339 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.636242    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-maas-api-tls\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.636425 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.636346    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxqfd\" (UniqueName: \"kubernetes.io/projected/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-kube-api-access-rxqfd\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.636425 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.636408    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-metrics-certs\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.675285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.675242    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" event={"ID":"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0","Type":"ContainerStarted","Data":"d65d7109fe0561a460663338f82d0584402ae983d434af833cf6fc1b0e75b3bd"}
Oct 30 00:51:38.677551 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.677518    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" event={"ID":"df652682-5a54-4e2e-afdb-25d2dffe9953","Type":"ContainerStarted","Data":"7d6a12a4cc5fd3376cb19898159af3df563ce59ed56cdf9188178006c057395e"}
Oct 30 00:51:38.737415 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.737339    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-maas-api-tls\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.738734 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.737685    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxqfd\" (UniqueName: \"kubernetes.io/projected/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-kube-api-access-rxqfd\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.738734 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.737747    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-metrics-certs\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.738734 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.737868    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.738734 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.738687    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.741355 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.741255    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-metrics-certs\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.741355 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.741311    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-maas-api-tls\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.759701 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.759670    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxqfd\" (UniqueName: \"kubernetes.io/projected/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-kube-api-access-rxqfd\") pod \"maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.801614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.801576    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"]
Oct 30 00:51:38.811745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.811717    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:38.825430 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.825383    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"]
Oct 30 00:51:38.828158 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.828129    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-3c2596\""
Oct 30 00:51:38.828347 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.828331    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-3c2596-dockercfg-zs4k5\""
Oct 30 00:51:38.839082 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.839053    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smx6j\" (UniqueName: \"kubernetes.io/projected/2ff68182-b017-41ea-890f-40309bb8c4c4-kube-api-access-smx6j\") pod \"payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:38.839290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.839105    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ff68182-b017-41ea-890f-40309bb8c4c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:38.869281 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.869249    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:38.916510 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.916469    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"]
Oct 30 00:51:38.925707 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.925676    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:38.947460 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.947416    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"]
Oct 30 00:51:38.948605 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.948571    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smx6j\" (UniqueName: \"kubernetes.io/projected/2ff68182-b017-41ea-890f-40309bb8c4c4-kube-api-access-smx6j\") pod \"payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:38.948672 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.948649    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ff68182-b017-41ea-890f-40309bb8c4c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:38.949790 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.949766    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ff68182-b017-41ea-890f-40309bb8c4c4-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:38.978783 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:38.978734    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smx6j\" (UniqueName: \"kubernetes.io/projected/2ff68182-b017-41ea-890f-40309bb8c4c4-kube-api-access-smx6j\") pod \"payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:39.050475 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.050168    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dbw6\" (UniqueName: \"kubernetes.io/projected/76c52766-11e5-45c9-8eaa-7de21df9f9ab-kube-api-access-2dbw6\") pod \"payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.050475 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.050298    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c52766-11e5-45c9-8eaa-7de21df9f9ab-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.134284 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.134248    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:39.151134 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.151092    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dbw6\" (UniqueName: \"kubernetes.io/projected/76c52766-11e5-45c9-8eaa-7de21df9f9ab-kube-api-access-2dbw6\") pod \"payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.151353 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.151204    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c52766-11e5-45c9-8eaa-7de21df9f9ab-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.151948 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.151924    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c52766-11e5-45c9-8eaa-7de21df9f9ab-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.173506 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.173479    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dbw6\" (UniqueName: \"kubernetes.io/projected/76c52766-11e5-45c9-8eaa-7de21df9f9ab-kube-api-access-2dbw6\") pod \"payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.219221 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.219193    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"]
Oct 30 00:51:39.261791 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.260808    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.444096 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.444066    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"]
Oct 30 00:51:39.446164 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:39.446126    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ff68182_b017_41ea_890f_40309bb8c4c4.slice/crio-843c65ccab00e6bdd3aaa5be34f3ee440939a9008c9654a2aece176720463932 WatchSource:0}: Error finding container 843c65ccab00e6bdd3aaa5be34f3ee440939a9008c9654a2aece176720463932: Status 404 returned error can't find the container with id 843c65ccab00e6bdd3aaa5be34f3ee440939a9008c9654a2aece176720463932
Oct 30 00:51:39.572670 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.572641    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"]
Oct 30 00:51:39.572759 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:39.572663    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76c52766_11e5_45c9_8eaa_7de21df9f9ab.slice/crio-a61d85997b27ccc4e7052f8ae9dd6cabf37cbd699a4e3e4a9e6ea107dc47af71 WatchSource:0}: Error finding container a61d85997b27ccc4e7052f8ae9dd6cabf37cbd699a4e3e4a9e6ea107dc47af71: Status 404 returned error can't find the container with id a61d85997b27ccc4e7052f8ae9dd6cabf37cbd699a4e3e4a9e6ea107dc47af71
Oct 30 00:51:39.685958 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.685915    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" event={"ID":"054a85f3-147d-469f-a4de-a5a29397d439","Type":"ContainerStarted","Data":"1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6"}
Oct 30 00:51:39.686523 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.686163    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:39.687835 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.687803    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" event={"ID":"863b2194-53c5-46ba-8f64-6215d4df058b","Type":"ContainerStarted","Data":"7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9"}
Oct 30 00:51:39.687973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.687922    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:39.689822 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.689797    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" event={"ID":"76c52766-11e5-45c9-8eaa-7de21df9f9ab","Type":"ContainerStarted","Data":"ceff8b422050a05b4722f832cdc7bac3a58e1c97634360d28dc4300cde569557"}
Oct 30 00:51:39.689961 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.689838    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" event={"ID":"76c52766-11e5-45c9-8eaa-7de21df9f9ab","Type":"ContainerStarted","Data":"a61d85997b27ccc4e7052f8ae9dd6cabf37cbd699a4e3e4a9e6ea107dc47af71"}
Oct 30 00:51:39.689961 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.689858    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:39.691766 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.691736    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw" event={"ID":"10be5e96-2dfe-4713-a2a4-36a42d7c6eba","Type":"ContainerStarted","Data":"37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a"}
Oct 30 00:51:39.691886 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.691786    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw" event={"ID":"10be5e96-2dfe-4713-a2a4-36a42d7c6eba","Type":"ContainerStarted","Data":"236e5ed0cf155e36d6115c82f45c2d7a9f05817c1420e19d5352cab3ee260817"}
Oct 30 00:51:39.692305 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.692271    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:39.696672 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.696646    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" event={"ID":"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0","Type":"ContainerStarted","Data":"6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2"}
Oct 30 00:51:39.696810 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.696786    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:39.698232 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.698209    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" event={"ID":"df652682-5a54-4e2e-afdb-25d2dffe9953","Type":"ContainerStarted","Data":"766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f"}
Oct 30 00:51:39.698344 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.698256    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:39.699743 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.699723    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" event={"ID":"2ff68182-b017-41ea-890f-40309bb8c4c4","Type":"ContainerStarted","Data":"a51764f7d3477c462489d0727108ca1d87ed2a7f1f03e7cbbf047b5898768bf2"}
Oct 30 00:51:39.699823 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.699761    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:39.699823 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.699783    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" event={"ID":"2ff68182-b017-41ea-890f-40309bb8c4c4","Type":"ContainerStarted","Data":"843c65ccab00e6bdd3aaa5be34f3ee440939a9008c9654a2aece176720463932"}
Oct 30 00:51:39.730559 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.730506    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" podStartSLOduration=2.111156315 podStartE2EDuration="4.730492052s" totalImagesPullingTime="2.619335737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:39.729564412 +0000 UTC m=+741.832318383" watchObservedRunningTime="2026-10-30 00:51:39.730492052 +0000 UTC m=+741.833246024"
Oct 30 00:51:39.772622 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.772564    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" podStartSLOduration=2.051657385 podStartE2EDuration="2.772549124s" totalImagesPullingTime="720.891739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:39.770644021 +0000 UTC m=+741.873398003" watchObservedRunningTime="2026-10-30 00:51:39.772549124 +0000 UTC m=+741.875303091"
Oct 30 00:51:39.777602 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.777568    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"]
Oct 30 00:51:39.781636 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.781617    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:39.797791 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.797768    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"]
Oct 30 00:51:39.800353 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.800319    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w5-da76a2\""
Oct 30 00:51:39.800453 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.800398    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w5-da76a2-dockercfg-fbxq4\""
Oct 30 00:51:39.814921 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.814877    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw" podStartSLOduration=1.474505881 podStartE2EDuration="1.814863477s" totalImagesPullingTime="340.357596ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:39.81303826 +0000 UTC m=+741.915792231" watchObservedRunningTime="2026-10-30 00:51:39.814863477 +0000 UTC m=+741.917617439"
Oct 30 00:51:39.837826 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.837796    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"]
Oct 30 00:51:39.841580 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.841565    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:39.856428 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.856374    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" podStartSLOduration=2.191720877 podStartE2EDuration="2.856355832s" totalImagesPullingTime="664.634955ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:39.853910441 +0000 UTC m=+741.956664414" watchObservedRunningTime="2026-10-30 00:51:39.856355832 +0000 UTC m=+741.959109803"
Oct 30 00:51:39.859142 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.859060    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g6tt\" (UniqueName: \"kubernetes.io/projected/db66c456-50da-4fa8-9361-24090e0fa74f-kube-api-access-2g6tt\") pod \"payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:39.859142 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.859119    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62bfv\" (UniqueName: \"kubernetes.io/projected/a414233b-c7f4-45bf-bb2a-837e6ae8203a-kube-api-access-62bfv\") pod \"payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:39.859500 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.859478    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a414233b-c7f4-45bf-bb2a-837e6ae8203a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:39.859562 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.859523    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66c456-50da-4fa8-9361-24090e0fa74f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:39.860347 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.860320    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"]
Oct 30 00:51:39.897002 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.896944    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" podStartSLOduration=1.466168788 podStartE2EDuration="3.896927612s" totalImagesPullingTime="2.430758824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:39.895819641 +0000 UTC m=+741.998573612" watchObservedRunningTime="2026-10-30 00:51:39.896927612 +0000 UTC m=+741.999681601"
Oct 30 00:51:39.939408 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.939354    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" podStartSLOduration=1.939339966 podStartE2EDuration="1.939339966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:39.937762588 +0000 UTC m=+742.040516570" watchObservedRunningTime="2026-10-30 00:51:39.939339966 +0000 UTC m=+742.042093936"
Oct 30 00:51:39.960535 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.960503    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g6tt\" (UniqueName: \"kubernetes.io/projected/db66c456-50da-4fa8-9361-24090e0fa74f-kube-api-access-2g6tt\") pod \"payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:39.960682 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.960546    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62bfv\" (UniqueName: \"kubernetes.io/projected/a414233b-c7f4-45bf-bb2a-837e6ae8203a-kube-api-access-62bfv\") pod \"payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:39.960766 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.960729    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a414233b-c7f4-45bf-bb2a-837e6ae8203a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:39.960898 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.960799    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66c456-50da-4fa8-9361-24090e0fa74f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:39.961420 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.961399    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a414233b-c7f4-45bf-bb2a-837e6ae8203a-plugins-config-volume\") pod \"payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:39.961504 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.961459    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66c456-50da-4fa8-9361-24090e0fa74f-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:39.982365 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.982324    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62bfv\" (UniqueName: \"kubernetes.io/projected/a414233b-c7f4-45bf-bb2a-837e6ae8203a-kube-api-access-62bfv\") pod \"payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") " pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:39.982365 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.982355    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g6tt\" (UniqueName: \"kubernetes.io/projected/db66c456-50da-4fa8-9361-24090e0fa74f-kube-api-access-2g6tt\") pod \"payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:39.986284 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:39.986239    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" podStartSLOduration=1.9862222 podStartE2EDuration="1.9862222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:39.984137483 +0000 UTC m=+742.086891453" watchObservedRunningTime="2026-10-30 00:51:39.9862222 +0000 UTC m=+742.088976168"
Oct 30 00:51:40.095475 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.095443    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:40.155213 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.154867    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:40.395031 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.394992    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"]
Oct 30 00:51:40.395404 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:40.395379    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb66c456_50da_4fa8_9361_24090e0fa74f.slice/crio-7a7876972ebcb92a94cf30ef8bac3a8db00a07061e7e0de31fddb7dd8aaf61b2 WatchSource:0}: Error finding container 7a7876972ebcb92a94cf30ef8bac3a8db00a07061e7e0de31fddb7dd8aaf61b2: Status 404 returned error can't find the container with id 7a7876972ebcb92a94cf30ef8bac3a8db00a07061e7e0de31fddb7dd8aaf61b2
Oct 30 00:51:40.459385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.459354    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"]
Oct 30 00:51:40.464798 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.464770    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.469285 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:40.469256    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda414233b_c7f4_45bf_bb2a_837e6ae8203a.slice/crio-89c09469b391b27ab8f7a7e5c10134c987ba54f9f5b937da09e48fa31ac34997 WatchSource:0}: Error finding container 89c09469b391b27ab8f7a7e5c10134c987ba54f9f5b937da09e48fa31ac34997: Status 404 returned error can't find the container with id 89c09469b391b27ab8f7a7e5c10134c987ba54f9f5b937da09e48fa31ac34997
Oct 30 00:51:40.471814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.471739    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"]
Oct 30 00:51:40.483921 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.483888    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"]
Oct 30 00:51:40.484385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.484161    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-7d2b75-serving-cert\""
Oct 30 00:51:40.566670 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.566637    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tshdb\" (UniqueName: \"kubernetes.io/projected/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-kube-api-access-tshdb\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.566860 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.566700    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-maas-api-tls\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.566860 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.566798    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-metrics-certs\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.566860 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.566828    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.667853 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.667768    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-maas-api-tls\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.667853 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.667821    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-metrics-certs\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.667853 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.667842    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.668087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.667908    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tshdb\" (UniqueName: \"kubernetes.io/projected/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-kube-api-access-tshdb\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.668818 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.668790    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.670602 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.670573    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-metrics-certs\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.670694 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.670574    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-maas-api-tls\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.688654 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.688622    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tshdb\" (UniqueName: \"kubernetes.io/projected/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-kube-api-access-tshdb\") pod \"maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:40.707927 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.707885    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" event={"ID":"db66c456-50da-4fa8-9361-24090e0fa74f","Type":"ContainerStarted","Data":"bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda"}
Oct 30 00:51:40.708122 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.707949    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" event={"ID":"db66c456-50da-4fa8-9361-24090e0fa74f","Type":"ContainerStarted","Data":"7a7876972ebcb92a94cf30ef8bac3a8db00a07061e7e0de31fddb7dd8aaf61b2"}
Oct 30 00:51:40.708122 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.707978    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:40.709567 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.709530    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" event={"ID":"a414233b-c7f4-45bf-bb2a-837e6ae8203a","Type":"ContainerStarted","Data":"01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01"}
Oct 30 00:51:40.709706 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.709586    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" event={"ID":"a414233b-c7f4-45bf-bb2a-837e6ae8203a","Type":"ContainerStarted","Data":"89c09469b391b27ab8f7a7e5c10134c987ba54f9f5b937da09e48fa31ac34997"}
Oct 30 00:51:40.710210 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.710186    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:40.751537 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.751481    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" podStartSLOduration=1.751467843 podStartE2EDuration="1.751467843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:40.750983112 +0000 UTC m=+742.853737093" watchObservedRunningTime="2026-10-30 00:51:40.751467843 +0000 UTC m=+742.854221810"
Oct 30 00:51:40.797191 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.797137    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" podStartSLOduration=1.7971240019999999 podStartE2EDuration="1.797124002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:40.795774745 +0000 UTC m=+742.898528755" watchObservedRunningTime="2026-10-30 00:51:40.797124002 +0000 UTC m=+742.899877974"
Oct 30 00:51:40.809222 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:40.809168    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:41.122724 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:41.122696    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"]
Oct 30 00:51:41.123688 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:41.123657    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5500c7ef_d6bc_4a85_b30d_b57e3e7af43d.slice/crio-2e19df1e804ca6d6f0b401303487c416af958a8f0a72ba0d94a778465ec42f32 WatchSource:0}: Error finding container 2e19df1e804ca6d6f0b401303487c416af958a8f0a72ba0d94a778465ec42f32: Status 404 returned error can't find the container with id 2e19df1e804ca6d6f0b401303487c416af958a8f0a72ba0d94a778465ec42f32
Oct 30 00:51:41.716887 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:41.716794    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd" event={"ID":"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d","Type":"ContainerStarted","Data":"423f8ea4dc7c89e7d421fba40660cd80c256f7264855b3c6d3f3751a87b9ad53"}
Oct 30 00:51:41.716887 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:41.716860    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd" event={"ID":"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d","Type":"ContainerStarted","Data":"2e19df1e804ca6d6f0b401303487c416af958a8f0a72ba0d94a778465ec42f32"}
Oct 30 00:51:41.717366 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:41.716893    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:41.758984 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:41.758919    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd" podStartSLOduration=1.471839294 podStartE2EDuration="1.758905127s" totalImagesPullingTime="287.065833ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:41.758088175 +0000 UTC m=+743.860842146" watchObservedRunningTime="2026-10-30 00:51:41.758905127 +0000 UTC m=+743.861659098"
Oct 30 00:51:42.135960 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.135930    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"]
Oct 30 00:51:42.139749 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.139734    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.160462 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.160433    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-b565ec\""
Oct 30 00:51:42.169373 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.169347    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"]
Oct 30 00:51:42.183657 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.183632    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5bdf78-c569-4818-ba87-17d49e069f30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.183800 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.183784    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfcm5\" (UniqueName: \"kubernetes.io/projected/af5bdf78-c569-4818-ba87-17d49e069f30-kube-api-access-zfcm5\") pod \"payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.219894 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.219863    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"]
Oct 30 00:51:42.226972 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.225499    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.255077 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.255049    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w4-b565ec-dockercfg-p7t4f\""
Oct 30 00:51:42.256618 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.256590    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"]
Oct 30 00:51:42.284561 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.284528    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01783257-1c00-4a08-9fc0-af40cf52d2e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.284561 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.284564    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsmqw\" (UniqueName: \"kubernetes.io/projected/01783257-1c00-4a08-9fc0-af40cf52d2e2-kube-api-access-zsmqw\") pod \"payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.284792 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.284636    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfcm5\" (UniqueName: \"kubernetes.io/projected/af5bdf78-c569-4818-ba87-17d49e069f30-kube-api-access-zfcm5\") pod \"payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.284792 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.284694    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5bdf78-c569-4818-ba87-17d49e069f30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.285411 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.285390    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5bdf78-c569-4818-ba87-17d49e069f30-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.310813 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.310769    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfcm5\" (UniqueName: \"kubernetes.io/projected/af5bdf78-c569-4818-ba87-17d49e069f30-kube-api-access-zfcm5\") pod \"payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.360930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.360882    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"]
Oct 30 00:51:42.365191 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.365144    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.385199 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.383464    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-3c2596-serving-cert\""
Oct 30 00:51:42.386318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.386233    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-maas-api-tls\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.386318 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.386286    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.386514 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.386360    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01783257-1c00-4a08-9fc0-af40cf52d2e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.386514 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.386387    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsmqw\" (UniqueName: \"kubernetes.io/projected/01783257-1c00-4a08-9fc0-af40cf52d2e2-kube-api-access-zsmqw\") pod \"payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.386514 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.386482    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-metrics-certs\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.386655 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.386533    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzkj6\" (UniqueName: \"kubernetes.io/projected/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-kube-api-access-zzkj6\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.387614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.387583    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01783257-1c00-4a08-9fc0-af40cf52d2e2-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.388219 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.388191    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"]
Oct 30 00:51:42.417109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.416165    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsmqw\" (UniqueName: \"kubernetes.io/projected/01783257-1c00-4a08-9fc0-af40cf52d2e2-kube-api-access-zsmqw\") pod \"payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.452467 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.452429    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:42.487152 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.487122    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-maas-api-tls\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.487152 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.487168    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.487428 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.487306    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-metrics-certs\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.487428 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.487355    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzkj6\" (UniqueName: \"kubernetes.io/projected/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-kube-api-access-zzkj6\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.488028 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.488002    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.490578 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.490547    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-maas-api-tls\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.492076 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.492048    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-metrics-certs\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.509585 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.509550    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzkj6\" (UniqueName: \"kubernetes.io/projected/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-kube-api-access-zzkj6\") pod \"maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.543558 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.543526    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:42.681490 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.681394    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:42.775852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.775804    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"]
Oct 30 00:51:42.777571 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:42.777537    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf5bdf78_c569_4818_ba87_17d49e069f30.slice/crio-736191b42baf8348c0101afbe2554e40a2e029fe4ac62b447cf12524a15bafcf WatchSource:0}: Error finding container 736191b42baf8348c0101afbe2554e40a2e029fe4ac62b447cf12524a15bafcf: Status 404 returned error can't find the container with id 736191b42baf8348c0101afbe2554e40a2e029fe4ac62b447cf12524a15bafcf
Oct 30 00:51:42.847793 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:42.847762    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01783257_1c00_4a08_9fc0_af40cf52d2e2.slice/crio-c3bc5df7bb49217cb65b5d1d563b8e2ceb50a8252a4b7a2f3a7dd2e710af57f3 WatchSource:0}: Error finding container c3bc5df7bb49217cb65b5d1d563b8e2ceb50a8252a4b7a2f3a7dd2e710af57f3: Status 404 returned error can't find the container with id c3bc5df7bb49217cb65b5d1d563b8e2ceb50a8252a4b7a2f3a7dd2e710af57f3
Oct 30 00:51:42.848419 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.848355    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"]
Oct 30 00:51:42.987109 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:42.987083    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"]
Oct 30 00:51:42.988865 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:42.988826    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71747683_1ef4_4e14_99a1_7c8d7cea3e5e.slice/crio-1f93b02bb5f215daca7a17e3fc333f8e3aa9d2a02021a5b8ddec1de287fe33da WatchSource:0}: Error finding container 1f93b02bb5f215daca7a17e3fc333f8e3aa9d2a02021a5b8ddec1de287fe33da: Status 404 returned error can't find the container with id 1f93b02bb5f215daca7a17e3fc333f8e3aa9d2a02021a5b8ddec1de287fe33da
Oct 30 00:51:43.731687 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.731587    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" event={"ID":"af5bdf78-c569-4818-ba87-17d49e069f30","Type":"ContainerStarted","Data":"da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729"}
Oct 30 00:51:43.731687 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.731644    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" event={"ID":"af5bdf78-c569-4818-ba87-17d49e069f30","Type":"ContainerStarted","Data":"736191b42baf8348c0101afbe2554e40a2e029fe4ac62b447cf12524a15bafcf"}
Oct 30 00:51:43.731687 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.731679    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:43.733157 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.733130    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" event={"ID":"01783257-1c00-4a08-9fc0-af40cf52d2e2","Type":"ContainerStarted","Data":"2d06bbe5c8011712b6744e732f508fcb006fd20f251613f10a9d233fd48c9a58"}
Oct 30 00:51:43.733319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.733204    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" event={"ID":"01783257-1c00-4a08-9fc0-af40cf52d2e2","Type":"ContainerStarted","Data":"c3bc5df7bb49217cb65b5d1d563b8e2ceb50a8252a4b7a2f3a7dd2e710af57f3"}
Oct 30 00:51:43.733319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.733238    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:43.734698 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.734673    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm" event={"ID":"71747683-1ef4-4e14-99a1-7c8d7cea3e5e","Type":"ContainerStarted","Data":"f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb"}
Oct 30 00:51:43.734790 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.734720    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm" event={"ID":"71747683-1ef4-4e14-99a1-7c8d7cea3e5e","Type":"ContainerStarted","Data":"1f93b02bb5f215daca7a17e3fc333f8e3aa9d2a02021a5b8ddec1de287fe33da"}
Oct 30 00:51:43.734790 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.734745    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:43.775985 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.775935    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" podStartSLOduration=1.775921162 podStartE2EDuration="1.775921162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:43.773698325 +0000 UTC m=+745.876452296" watchObservedRunningTime="2026-10-30 00:51:43.775921162 +0000 UTC m=+745.878675132"
Oct 30 00:51:43.817638 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.817588    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm" podStartSLOduration=1.451368781 podStartE2EDuration="1.817574581s" totalImagesPullingTime="366.2058ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:43.816734412 +0000 UTC m=+745.919488382" watchObservedRunningTime="2026-10-30 00:51:43.817574581 +0000 UTC m=+745.920328551"
Oct 30 00:51:43.860437 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:43.860378    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" podStartSLOduration=1.8603646440000001 podStartE2EDuration="1.860364644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:43.858508724 +0000 UTC m=+745.961262694" watchObservedRunningTime="2026-10-30 00:51:43.860364644 +0000 UTC m=+745.963118632"
Oct 30 00:51:44.352698 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.352662    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"]
Oct 30 00:51:44.356750 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.356734    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.376924 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.376894    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"]
Oct 30 00:51:44.387966 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.387928    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-da76a2-serving-cert\""
Oct 30 00:51:44.410203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.410145    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-maas-api-tls\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.410363 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.410250    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd1152b0-ccfa-4507-b220-231ee84d3790-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.410363 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.410269    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-metrics-certs\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.410363 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.410299    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frzf4\" (UniqueName: \"kubernetes.io/projected/bd1152b0-ccfa-4507-b220-231ee84d3790-kube-api-access-frzf4\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.511021 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.510985    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd1152b0-ccfa-4507-b220-231ee84d3790-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.511021 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.511022    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-metrics-certs\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.511324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.511060    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frzf4\" (UniqueName: \"kubernetes.io/projected/bd1152b0-ccfa-4507-b220-231ee84d3790-kube-api-access-frzf4\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.511324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.511252    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-maas-api-tls\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.511762 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.511740    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd1152b0-ccfa-4507-b220-231ee84d3790-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.513695 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.513677    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-maas-api-tls\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.513816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.513799    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-metrics-certs\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.532608 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.532579    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frzf4\" (UniqueName: \"kubernetes.io/projected/bd1152b0-ccfa-4507-b220-231ee84d3790-kube-api-access-frzf4\") pod \"maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.670713 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.670628    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:44.973495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:44.973466    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"]
Oct 30 00:51:44.973822 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:44.973734    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd1152b0_ccfa_4507_b220_231ee84d3790.slice/crio-961492def86ccd4978268f732166165a19de5d878ebbf658c09f5788a6987007 WatchSource:0}: Error finding container 961492def86ccd4978268f732166165a19de5d878ebbf658c09f5788a6987007: Status 404 returned error can't find the container with id 961492def86ccd4978268f732166165a19de5d878ebbf658c09f5788a6987007
Oct 30 00:51:45.754203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:45.754074    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln" event={"ID":"bd1152b0-ccfa-4507-b220-231ee84d3790","Type":"ContainerStarted","Data":"13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3"}
Oct 30 00:51:45.754203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:45.754139    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln" event={"ID":"bd1152b0-ccfa-4507-b220-231ee84d3790","Type":"ContainerStarted","Data":"961492def86ccd4978268f732166165a19de5d878ebbf658c09f5788a6987007"}
Oct 30 00:51:45.754432 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:45.754220    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:45.799080 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:45.799013    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln" podStartSLOduration=1.487883833 podStartE2EDuration="1.798998652s" totalImagesPullingTime="311.114819ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:45.7968891 +0000 UTC m=+747.899643070" watchObservedRunningTime="2026-10-30 00:51:45.798998652 +0000 UTC m=+747.901752664"
Oct 30 00:51:46.722606 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:46.722572    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 00:51:47.729187 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:47.729135    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 00:51:48.240567 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.240526    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"]
Oct 30 00:51:48.244781 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.244759    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.263293 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.263268    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"]
Oct 30 00:51:48.274479 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.274446    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-b565ec-serving-cert\""
Oct 30 00:51:48.363476 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.363436    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.363635 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.363510    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-metrics-certs\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.363684 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.363659    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.363816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.363796    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ll85\" (UniqueName: \"kubernetes.io/projected/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-kube-api-access-8ll85\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.464695 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.464661    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-metrics-certs\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.464850 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.464718    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.464850 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.464774    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ll85\" (UniqueName: \"kubernetes.io/projected/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-kube-api-access-8ll85\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.464850 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.464807    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.464979 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:51:48.464929    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-serving-cert: secret "maas-api-e2e-worker-w4-b565ec-serving-cert" not found
Oct 30 00:51:48.465042 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:51:48.465028    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls podName:d0dac3d5-db84-4143-8347-ce29d3cf0d2e nodeName:}" failed. No retries permitted until 2026-10-30 00:51:48.965004257 +0000 UTC m=+751.067758219 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls") pod "maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft" (UID: "d0dac3d5-db84-4143-8347-ce29d3cf0d2e") : secret "maas-api-e2e-worker-w4-b565ec-serving-cert" not found
Oct 30 00:51:48.465504 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.465481    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.467411 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.467380    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-metrics-certs\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.486071 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.486038    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ll85\" (UniqueName: \"kubernetes.io/projected/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-kube-api-access-8ll85\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.970309 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.970269    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:48.972824 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:48.972805    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:49.159998 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:49.159946    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:49.464800 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:49.464771    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"]
Oct 30 00:51:49.464902 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:49.464804    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0dac3d5_db84_4143_8347_ce29d3cf0d2e.slice/crio-b9b970db3c77b1268d939a52558a3b8c86acd299791e03f26c3814c56018a0d7 WatchSource:0}: Error finding container b9b970db3c77b1268d939a52558a3b8c86acd299791e03f26c3814c56018a0d7: Status 404 returned error can't find the container with id b9b970db3c77b1268d939a52558a3b8c86acd299791e03f26c3814c56018a0d7
Oct 30 00:51:49.754964 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:49.754876    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 00:51:49.780213 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:49.780159    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft" event={"ID":"d0dac3d5-db84-4143-8347-ce29d3cf0d2e","Type":"ContainerStarted","Data":"b9b970db3c77b1268d939a52558a3b8c86acd299791e03f26c3814c56018a0d7"}
Oct 30 00:51:50.713220 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.713189    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 00:51:50.713911 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.713278    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 00:51:50.714197 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.714149    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 00:51:50.714644 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.714614    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 00:51:50.714764 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.714696    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 00:51:50.714764 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.714746    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 00:51:50.790199 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.790139    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft" event={"ID":"d0dac3d5-db84-4143-8347-ce29d3cf0d2e","Type":"ContainerStarted","Data":"8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a"}
Oct 30 00:51:50.790380 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:50.790345    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:51.060700 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:51.060645    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft" podStartSLOduration=2.656184579 podStartE2EDuration="3.060631443s" totalImagesPullingTime="404.446864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:51.059125711 +0000 UTC m=+753.161879680" watchObservedRunningTime="2026-10-30 00:51:51.060631443 +0000 UTC m=+753.163385414"
Oct 30 00:51:51.720371 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:51.720338    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 00:51:51.766469 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:51.766442    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 00:51:52.726926 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:52.726891    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 00:51:54.751935 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:54.751907    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 00:51:54.752559 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:54.752533    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 00:51:56.027376 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.027345    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"]
Oct 30 00:51:56.031708 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.031691    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.055457 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.055432    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-298ddc-serving-cert\""
Oct 30 00:51:56.070434 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.070402    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"]
Oct 30 00:51:56.151504 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.151473    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfqbj\" (UniqueName: \"kubernetes.io/projected/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-kube-api-access-rfqbj\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.151682 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.151626    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-metrics-certs\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.151765 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.151691    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.151765 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.151725    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.252877 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.252839    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-metrics-certs\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.253054 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.252895    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.253054 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.252941    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.253054 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.253038    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfqbj\" (UniqueName: \"kubernetes.io/projected/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-kube-api-access-rfqbj\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.253371 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:51:56.253342    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-serving-cert: secret "maas-api-e2e-worker-w3-298ddc-serving-cert" not found
Oct 30 00:51:56.253517 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:51:56.253477    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls podName:64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf nodeName:}" failed. No retries permitted until 2026-10-30 00:51:56.753456977 +0000 UTC m=+758.856210929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls") pod "maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6" (UID: "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf") : secret "maas-api-e2e-worker-w3-298ddc-serving-cert" not found
Oct 30 00:51:56.253748 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.253723    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.255526 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.255500    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-metrics-certs\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.276905 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.276874    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfqbj\" (UniqueName: \"kubernetes.io/projected/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-kube-api-access-rfqbj\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.758310 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.758273    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.760801 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.760773    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls\") pod \"maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:56.802709 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.802676    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 00:51:56.947208 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:56.947147    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:57.246991 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:57.246964    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"]
Oct 30 00:51:57.247472 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:51:57.247451    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64ec0ce9_af7f_4d1a_9762_b7f5c55b66bf.slice/crio-311bfbd3f5e068ff2f0fd9aafc2f1d836b0e5608632db8492edcb14440953e12 WatchSource:0}: Error finding container 311bfbd3f5e068ff2f0fd9aafc2f1d836b0e5608632db8492edcb14440953e12: Status 404 returned error can't find the container with id 311bfbd3f5e068ff2f0fd9aafc2f1d836b0e5608632db8492edcb14440953e12
Oct 30 00:51:57.840728 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:57.840688    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6" event={"ID":"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf","Type":"ContainerStarted","Data":"467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c"}
Oct 30 00:51:57.840920 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:57.840788    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:51:57.840920 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:57.840807    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6" event={"ID":"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf","Type":"ContainerStarted","Data":"311bfbd3f5e068ff2f0fd9aafc2f1d836b0e5608632db8492edcb14440953e12"}
Oct 30 00:51:57.885755 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:51:57.885621    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6" podStartSLOduration=2.53780051 podStartE2EDuration="2.885605991s" totalImagesPullingTime="347.805481ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:51:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:51:57.882984756 +0000 UTC m=+759.985738724" watchObservedRunningTime="2026-10-30 00:51:57.885605991 +0000 UTC m=+759.988359961"
Oct 30 00:51:57.911476 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:51:57.911432    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:52:03.852319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:03.852290    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 00:52:15.620760 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.620722    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"]
Oct 30 00:52:15.630106 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.630080    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.645239 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.645209    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"]
Oct 30 00:52:15.662105 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662068    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbxkf\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-kube-api-access-kbxkf\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662284 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662219    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662284 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662269    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662372 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662356    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662484 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662469    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662526 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662504    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662611 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662596    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662669 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662655    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.662775 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.662759    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.734676 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.734645    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-5a344d-openshift-default-dockercfg-7z8cg\""
Oct 30 00:52:15.763903 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.763876    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764063 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.763918    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbxkf\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-kube-api-access-kbxkf\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764063 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.763949    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764063 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.763969    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764279 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764107    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764279 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764163    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764279 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764216    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764279 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764257    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764489 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764389    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764489 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764407    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764596 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764583    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764690 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764665    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764759 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764738    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.764839 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.764821    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.766448 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.766421    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.766744 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.766728    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.786264 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.786242    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbxkf\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-kube-api-access-kbxkf\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.786531 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.786509    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:15.949399 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:15.949311    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:16.271870 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:16.271839    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"]
Oct 30 00:52:16.272631 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:52:16.272599    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb6ea9dd_a956_4770_a00c_8dd00370ee41.slice/crio-59553f4538fdfe90ab4586fc978457b88bcd4187cb1f57ccc5f75a6d63f6738a WatchSource:0}: Error finding container 59553f4538fdfe90ab4586fc978457b88bcd4187cb1f57ccc5f75a6d63f6738a: Status 404 returned error can't find the container with id 59553f4538fdfe90ab4586fc978457b88bcd4187cb1f57ccc5f75a6d63f6738a
Oct 30 00:52:16.275970 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:16.275935    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:52:16.276082 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:16.276013    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:52:16.276082 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:16.276046    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:52:16.955092 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:16.955054    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" event={"ID":"cb6ea9dd-a956-4770-a00c-8dd00370ee41","Type":"ContainerStarted","Data":"349c5523205c67762bb034150aad22a459ed5bb1f591f7a1dd9b2f4224a71782"}
Oct 30 00:52:16.955530 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:16.955109    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" event={"ID":"cb6ea9dd-a956-4770-a00c-8dd00370ee41","Type":"ContainerStarted","Data":"59553f4538fdfe90ab4586fc978457b88bcd4187cb1f57ccc5f75a6d63f6738a"}
Oct 30 00:52:17.017414 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:17.017364    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" podStartSLOduration=2.017349614 podStartE2EDuration="2.017349614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:52:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:52:17.015567784 +0000 UTC m=+779.118321772" watchObservedRunningTime="2026-10-30 00:52:17.017349614 +0000 UTC m=+779.120103589"
Oct 30 00:52:17.961202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:17.951316    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:17.961202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:17.959285    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 00:52:17.961202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:17.959343    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 00:52:18.962208 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:18.958647    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 00:52:18.962208 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:18.958712    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 00:52:19.959198 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:19.958289    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 00:52:19.959198 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:19.958349    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 00:52:20.964632 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:20.964597    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 00:52:20.965102 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:20.964658    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 00:52:21.960202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:21.959293    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 00:52:21.960202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:21.959360    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 00:52:22.954626 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:22.954594    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:22.955032 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:22.954890    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:22.955721 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:52:22.955701    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 00:52:27.910796 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:52:27.910754    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:52:57.910139 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:52:57.910057    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:53:00.741551 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.741517    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"]
Oct 30 00:53:00.745857 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.745832    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.779746 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.779702    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"]
Oct 30 00:53:00.895734 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.895691    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gjt5\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-kube-api-access-4gjt5\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.895913 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.895755    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.895913 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.895843    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.895913 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.895887    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.896050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.895973    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.896050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.896010    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.896050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.896036    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.896152 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.896109    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.896272 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.896255    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997278 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997165    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997278 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997240    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997278 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997270    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997296    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997318    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997339    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997357    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997434    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997535    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gjt5\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-kube-api-access-4gjt5\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997876 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997744    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997876 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997769    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.997988 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.997967    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.998039 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.998026    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:00.998125 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:00.998106    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:01.000231 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.000208    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:01.000353 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.000337    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:01.018346 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.018322    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:01.018493 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.018466    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gjt5\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-kube-api-access-4gjt5\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:01.063647 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.063615    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:01.380227 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:53:01.380197    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1004f33b_3dbe_4036_8e4e_6d6bf76a4a85.slice/crio-712f4443c2384066864716b3b3648d77ef9d029946d51d2c43465dd5e606d23e WatchSource:0}: Error finding container 712f4443c2384066864716b3b3648d77ef9d029946d51d2c43465dd5e606d23e: Status 404 returned error can't find the container with id 712f4443c2384066864716b3b3648d77ef9d029946d51d2c43465dd5e606d23e
Oct 30 00:53:01.380337 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.380244    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"]
Oct 30 00:53:01.382499 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.382469    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:01.382552 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.382534    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:01.382587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:01.382564    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:02.271781 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:02.271736    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" event={"ID":"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85","Type":"ContainerStarted","Data":"c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3"}
Oct 30 00:53:02.272235 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:02.271796    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" event={"ID":"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85","Type":"ContainerStarted","Data":"712f4443c2384066864716b3b3648d77ef9d029946d51d2c43465dd5e606d23e"}
Oct 30 00:53:02.359201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:02.355681    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" podStartSLOduration=2.355661112 podStartE2EDuration="2.355661112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:02.327088482 +0000 UTC m=+824.429842452" watchObservedRunningTime="2026-10-30 00:53:02.355661112 +0000 UTC m=+824.458415085"
Oct 30 00:53:03.065405 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:03.065319    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:03.067078 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:03.067041    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 00:53:03.067262 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:03.067107    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 00:53:04.064607 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:04.064569    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 00:53:04.065066 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:04.064726    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 00:53:05.064617 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:05.064580    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 00:53:05.065118 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:05.064669    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 00:53:06.064565 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:06.064528    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 00:53:06.064764 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:06.064609    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 00:53:07.064696 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:07.064657    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 00:53:07.064911 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:07.064737    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 00:53:08.068671 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:08.068642    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:08.069128 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:08.069106    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:08.069845 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:08.069830    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 00:53:14.982384 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:14.982352    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"]
Oct 30 00:53:14.986636 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:14.986616    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.016275 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.016249    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-7d2b75-openshift-default-dockercfg-pft72\""
Oct 30 00:53:15.018852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.018831    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"]
Oct 30 00:53:15.087009 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.086980    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbszb\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-kube-api-access-vbszb\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087221 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087022    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-podinfo\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087221 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087058    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-token\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087221 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087130    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-envoy\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087367 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087257    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-credential-socket\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087412 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087383    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a1a771f-88ba-49b1-89fa-630dee6e723f-istiod-ca-cert\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087501 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087485    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-socket\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087591 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087578    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-data\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.087672 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.087659    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-certs\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.188719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.188679    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-credential-socket\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.188719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.188720    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a1a771f-88ba-49b1-89fa-630dee6e723f-istiod-ca-cert\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.188973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.188776    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-socket\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.188973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.188816    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-data\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.188973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.188863    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-certs\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.188973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.188889    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbszb\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-kube-api-access-vbszb\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.188973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.188926    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-podinfo\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.189272 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.189167    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-credential-socket\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.189272 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.189240    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-socket\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.189272 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.189245    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-token\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.189446 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.189290    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-envoy\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.189446 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.189348    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-certs\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.189560 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.189471    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-data\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.189684 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.189662    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a1a771f-88ba-49b1-89fa-630dee6e723f-istiod-ca-cert\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.191472 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.191450    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-envoy\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.191592 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.191573    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-podinfo\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.209385 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.209362    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbszb\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-kube-api-access-vbszb\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.209757 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.209737    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-token\") pod \"e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") " pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.300820 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.300792    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:15.615319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.615291    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"]
Oct 30 00:53:15.616111 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:53:15.616081    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1a771f_88ba_49b1_89fa_630dee6e723f.slice/crio-59812e46cb592897fc4517e4a8c235e1408528a8fe5342d7cc7b91263f2aaa5f WatchSource:0}: Error finding container 59812e46cb592897fc4517e4a8c235e1408528a8fe5342d7cc7b91263f2aaa5f: Status 404 returned error can't find the container with id 59812e46cb592897fc4517e4a8c235e1408528a8fe5342d7cc7b91263f2aaa5f
Oct 30 00:53:15.618794 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.618753    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:15.618874 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.618848    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:15.618915 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.618897    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:15.771797 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.771767    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"]
Oct 30 00:53:15.775800 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.775778    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.793399 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.793368    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"]
Oct 30 00:53:15.795353 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795318    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795506 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795369    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795555 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795524    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795594 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795581    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795632 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795607    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795670 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795638    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795703    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795773 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795758    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvk2x\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-kube-api-access-wvk2x\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.795968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.795802    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.897535 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897427    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.897535 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897504    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.897780 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897536    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.897780 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897572    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.897780 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897636    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.897780 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897691    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvk2x\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-kube-api-access-wvk2x\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.897780 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897733    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.898031 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897795    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.898031 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897826    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.898031 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897925    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-credential-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.898031 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.897966    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-data\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.898031 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.898003    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-certs\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.898320 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.898259    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-socket\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.899037 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.899010    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istiod-ca-cert\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.900513 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.900491    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-podinfo\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.900596 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.900580    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-envoy\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.917956 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.917923    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-token\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:15.918089 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:15.918032    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvk2x\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-kube-api-access-wvk2x\") pod \"e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") " pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:16.090386 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.090352    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:16.366614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.366583    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" event={"ID":"9a1a771f-88ba-49b1-89fa-630dee6e723f","Type":"ContainerStarted","Data":"333b210dd1c1831fcd62326dd0b2ad65e031b54b08125dd66111d8be214eef15"}
Oct 30 00:53:16.366801 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.366643    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" event={"ID":"9a1a771f-88ba-49b1-89fa-630dee6e723f","Type":"ContainerStarted","Data":"59812e46cb592897fc4517e4a8c235e1408528a8fe5342d7cc7b91263f2aaa5f"}
Oct 30 00:53:16.403692 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:53:16.403664    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e2a58fe_562d_4d20_9a12_fa42928bc9d2.slice/crio-e26e106e1f2472851fd5935079a5eebb554ea6e6dcfc7e9f09786991f73da50f WatchSource:0}: Error finding container e26e106e1f2472851fd5935079a5eebb554ea6e6dcfc7e9f09786991f73da50f: Status 404 returned error can't find the container with id e26e106e1f2472851fd5935079a5eebb554ea6e6dcfc7e9f09786991f73da50f
Oct 30 00:53:16.403956 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.403933    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"]
Oct 30 00:53:16.406079 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.406042    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:16.406158 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.406124    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:16.406217 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.406168    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:16.412269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:16.412225    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podStartSLOduration=2.412213053 podStartE2EDuration="2.412213053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:16.410597616 +0000 UTC m=+838.513351587" watchObservedRunningTime="2026-10-30 00:53:16.412213053 +0000 UTC m=+838.514967024"
Oct 30 00:53:17.301576 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:17.301540    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:17.303481 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:17.303452    6011 patch_prober.go:29] interesting pod/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 00:53:17.303640 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:17.303526    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 00:53:17.399128 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:17.399086    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" event={"ID":"2e2a58fe-562d-4d20-9a12-fa42928bc9d2","Type":"ContainerStarted","Data":"717dd57e7a9cfdbe9f281fb6e2916163186408fe9113e6617707a7258b8bcbee"}
Oct 30 00:53:17.399303 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:17.399154    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" event={"ID":"2e2a58fe-562d-4d20-9a12-fa42928bc9d2","Type":"ContainerStarted","Data":"e26e106e1f2472851fd5935079a5eebb554ea6e6dcfc7e9f09786991f73da50f"}
Oct 30 00:53:17.473202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:17.472333    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podStartSLOduration=2.472312789 podStartE2EDuration="2.472312789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:17.469709803 +0000 UTC m=+839.572463775" watchObservedRunningTime="2026-10-30 00:53:17.472312789 +0000 UTC m=+839.575066779"
Oct 30 00:53:18.096290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:18.095224    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:18.096290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:18.095341    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 00:53:18.096290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:18.095385    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 00:53:18.302618 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:18.302581    6011 patch_prober.go:29] interesting pod/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 00:53:18.303223 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:18.303190    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 00:53:19.092203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:19.092118    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 00:53:19.092388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:19.092199    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 00:53:19.307965 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:19.307928    6011 patch_prober.go:29] interesting pod/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 00:53:19.308583 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:19.308548    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 00:53:20.096032 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:20.094721    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 00:53:20.096032 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:20.094809    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 00:53:20.301520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:20.301477    6011 patch_prober.go:29] interesting pod/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 00:53:20.301679 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:20.301544    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 00:53:21.113503 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:21.113467    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 00:53:21.114086 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:21.114057    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 00:53:21.301702 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:21.301667    6011 patch_prober.go:29] interesting pod/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 00:53:21.301873 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:21.301723    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 00:53:22.091315 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:22.091266    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 00:53:22.091501 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:22.091332    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 00:53:22.301458 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:22.301412    6011 patch_prober.go:29] interesting pod/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Oct 30 00:53:22.301848 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:22.301480    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Oct 30 00:53:23.306082 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:23.306048    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:23.306779 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:23.306754    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:23.307765 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:23.307743    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 00:53:24.092605 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:24.092566    6011 patch_prober.go:29] interesting pod/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:53:24.092825 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:24.092626    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:53:24.094145 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:24.094124    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:24.094667 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:24.094647    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:24.095357 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:24.095338    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 00:53:27.912562 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:53:27.912519    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:53:45.202715 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.202673    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"]
Oct 30 00:53:45.208317 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.208292    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.239339 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.239303    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-8cf33a-openshift-default-dockercfg-j7d2l\""
Oct 30 00:53:45.243793 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.243763    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"]
Oct 30 00:53:45.245117 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245090    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpmpk\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-kube-api-access-tpmpk\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245137    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245301 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245264    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-credential-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245301 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245294    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-data\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245404 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245389    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-certs\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245449 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245432    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-envoy\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245528 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245511    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-token\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245587 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245565    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-podinfo\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.245667 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.245651    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istiod-ca-cert\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.346937 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.346907    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istiod-ca-cert\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347121 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.346970    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpmpk\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-kube-api-access-tpmpk\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347121 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.346998    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347121 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347022    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-credential-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347121 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347039    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-data\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347121 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347114    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-certs\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347575 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347145    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-envoy\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347575 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347234    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-token\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347575 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347260    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-podinfo\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347780 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347754    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istiod-ca-cert\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347894 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347824    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-data\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.347955 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347892    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-certs\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.348017 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.347984    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-credential-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.348247 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.348227    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.349770 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.349752    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-envoy\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.350274 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.350256    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-podinfo\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.371714 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.371682    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-token\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.372287 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.372266    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpmpk\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-kube-api-access-tpmpk\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.536875 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.536836    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:45.854786 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:53:45.854753    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d158e9_6fc5_4b2f_82a7_b5cda7f85187.slice/crio-bfbc17d0a2d8f7dd2ceea5eb855c3f99fd2b776af70c061d5dbc2d33cc2483f5 WatchSource:0}: Error finding container bfbc17d0a2d8f7dd2ceea5eb855c3f99fd2b776af70c061d5dbc2d33cc2483f5: Status 404 returned error can't find the container with id bfbc17d0a2d8f7dd2ceea5eb855c3f99fd2b776af70c061d5dbc2d33cc2483f5
Oct 30 00:53:45.855085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.855061    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"]
Oct 30 00:53:45.857472 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.857437    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:45.857589 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.857504    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:45.857589 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:45.857534    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:53:46.666885 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:46.666844    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" event={"ID":"11d158e9-6fc5-4b2f-82a7-b5cda7f85187","Type":"ContainerStarted","Data":"96835eacd10338c7e7fa395ae015b61338ca5b9c829724f1e97e6f783eab410d"}
Oct 30 00:53:46.667340 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:46.666909    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" event={"ID":"11d158e9-6fc5-4b2f-82a7-b5cda7f85187","Type":"ContainerStarted","Data":"bfbc17d0a2d8f7dd2ceea5eb855c3f99fd2b776af70c061d5dbc2d33cc2483f5"}
Oct 30 00:53:47.539205 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:47.537528    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:47.550201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:47.547483    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:53:47.550201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:47.547540    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:53:48.537634 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:48.537601    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:53:48.538024 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:48.537665    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:53:49.538089 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:49.538053    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:53:49.538589 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:49.538122    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:53:50.537487 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:50.537452    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:53:50.537675 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:50.537525    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:53:51.537942 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:51.537906    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 00:53:51.538347 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:51.537988    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 00:53:52.646325 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:52.646287    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:52.651200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:52.647822    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:52.651200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:52.650619    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 00:53:52.705079 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:53:52.705025    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podStartSLOduration=7.705009194 podStartE2EDuration="7.705009194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:53:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:53:46.7142499 +0000 UTC m=+868.817003889" watchObservedRunningTime="2026-10-30 00:53:52.705009194 +0000 UTC m=+874.807763164"
Oct 30 00:53:57.910623 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:53:57.910585    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:54:01.055989 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.055956    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"]
Oct 30 00:54:01.061914 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.061892    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.081046 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.081016    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"]
Oct 30 00:54:01.090521 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.090484    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-298ddc-openshift-default-dockercfg-lcv79\""
Oct 30 00:54:01.159090 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159051    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-data\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159311 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159102    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-token\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159311 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159281    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-envoy\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159388 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159345    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-socket\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159434 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159415    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-credential-socket\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159473 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159443    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-podinfo\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159473 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159464    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqqnm\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-kube-api-access-rqqnm\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159559 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159546    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istiod-ca-cert\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.159637 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.159614    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-certs\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.260922 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.260880    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-data\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261139 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.260933    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-token\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261139 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261012    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-envoy\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261139 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261068    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-socket\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261139 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261101    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-credential-socket\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261139 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261127    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-podinfo\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261157    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqqnm\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-kube-api-access-rqqnm\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261242    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istiod-ca-cert\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261318    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-certs\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261382    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-data\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261520 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261464    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-socket\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261688    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-certs\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.261852 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.261756    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-credential-socket\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.262075 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.262052    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istiod-ca-cert\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.263744 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.263724    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-podinfo\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.264595 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.264572    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-envoy\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.280848 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.280820    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-token\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.281213 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.281190    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqqnm\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-kube-api-access-rqqnm\") pod \"e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") " pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.377197 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.377105    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:01.690578 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.690545    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"]
Oct 30 00:54:01.690715 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:54:01.690639    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a877407_b50a_41e5_91af_7d9cf4d2fe53.slice/crio-b2858af64e493382a17c058df183f981a637358f61d416bce536c85456518279 WatchSource:0}: Error finding container b2858af64e493382a17c058df183f981a637358f61d416bce536c85456518279: Status 404 returned error can't find the container with id b2858af64e493382a17c058df183f981a637358f61d416bce536c85456518279
Oct 30 00:54:01.693214 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.693153    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:01.693282 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.693251    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:01.693315 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.693292    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:01.775801 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.775762    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" event={"ID":"9a877407-b50a-41e5-91af-7d9cf4d2fe53","Type":"ContainerStarted","Data":"cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e"}
Oct 30 00:54:01.775971 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.775826    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" event={"ID":"9a877407-b50a-41e5-91af-7d9cf4d2fe53","Type":"ContainerStarted","Data":"b2858af64e493382a17c058df183f981a637358f61d416bce536c85456518279"}
Oct 30 00:54:01.819059 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:01.818995    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podStartSLOduration=0.818977178 podStartE2EDuration="818.977178ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:01.817386724 +0000 UTC m=+883.920140699" watchObservedRunningTime="2026-10-30 00:54:01.818977178 +0000 UTC m=+883.921731151"
Oct 30 00:54:02.378277 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:02.378230    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:02.380147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:02.380119    6011 patch_prober.go:29] interesting pod/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:54:02.380299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:02.380223    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:54:03.377814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:03.377775    6011 patch_prober.go:29] interesting pod/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:54:03.378024 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:03.377850    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:54:04.377736 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:04.377701    6011 patch_prober.go:29] interesting pod/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:54:04.378162 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:04.377756    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:54:05.378486 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:05.378454    6011 patch_prober.go:29] interesting pod/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:54:05.378913 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:05.378508    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:54:06.377869 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:06.377830    6011 patch_prober.go:29] interesting pod/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:54:06.378060 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:06.377885    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:54:07.378287 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:07.378251    6011 patch_prober.go:29] interesting pod/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Oct 30 00:54:07.378734 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:07.378311    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Oct 30 00:54:08.528108 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:08.524547    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:08.528108 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:08.525942    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:08.528951 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:08.528934    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 00:54:11.245373 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:11.245322    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:54:11.246005 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:11.245591    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:54:13.263450 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.263412    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"]
Oct 30 00:54:13.267770 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.267745    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.283578 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.283546    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"]
Oct 30 00:54:13.293622 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.293591    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-3c2596\"/\"openshift-service-ca.crt\""
Oct 30 00:54:13.293767 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.293699    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-3c2596\"/\"kube-root-ca.crt\""
Oct 30 00:54:13.293820 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.293790    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-3c2596\"/\"default-dockercfg-jgw45\""
Oct 30 00:54:13.296821 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.296804    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-3c2596\"/\"distinct-w0-3c2596-kserve-self-signed-certs\""
Oct 30 00:54:13.389389 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.389352    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-dshm\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.389576 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.389414    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-home\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.389576 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.389530    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tmp-dir\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.389683 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.389662    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrpmc\" (UniqueName: \"kubernetes.io/projected/1622311a-a544-4ca5-bb79-ed4c0945b8bf-kube-api-access-qrpmc\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.389731 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.389717    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-model-cache\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.389765 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.389742    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tls-certs\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.491105 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.491069    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-dshm\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.491354 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.491133    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-home\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.491354 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.491217    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tmp-dir\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.491354 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.491328    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrpmc\" (UniqueName: \"kubernetes.io/projected/1622311a-a544-4ca5-bb79-ed4c0945b8bf-kube-api-access-qrpmc\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.491545 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.491393    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-model-cache\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.491545 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.491419    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tls-certs\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.491989 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.491960    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-home\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.492315 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.492258    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tmp-dir\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.492315 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.492299    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-model-cache\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.493712 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.493689    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-dshm\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.494084 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.494063    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tls-certs\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.513440 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.513346    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrpmc\" (UniqueName: \"kubernetes.io/projected/1622311a-a544-4ca5-bb79-ed4c0945b8bf-kube-api-access-qrpmc\") pod \"distinct-w0-3c2596-kserve-6df759769c-2zfhr\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") " pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.585790 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.585761    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:13.884990 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.884945    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"]
Oct 30 00:54:13.885278 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:54:13.885251    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1622311a_a544_4ca5_bb79_ed4c0945b8bf.slice/crio-118e52aafbcc3263459cd93c2e2f93eee6d176ba33f28ccafdfc1a8e9e003d4d WatchSource:0}: Error finding container 118e52aafbcc3263459cd93c2e2f93eee6d176ba33f28ccafdfc1a8e9e003d4d: Status 404 returned error can't find the container with id 118e52aafbcc3263459cd93c2e2f93eee6d176ba33f28ccafdfc1a8e9e003d4d
Oct 30 00:54:13.907415 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:13.907387    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" event={"ID":"1622311a-a544-4ca5-bb79-ed4c0945b8bf","Type":"ContainerStarted","Data":"118e52aafbcc3263459cd93c2e2f93eee6d176ba33f28ccafdfc1a8e9e003d4d"}
Oct 30 00:54:14.915271 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:14.915219    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" event={"ID":"1622311a-a544-4ca5-bb79-ed4c0945b8bf","Type":"ContainerStarted","Data":"26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12"}
Oct 30 00:54:14.967757 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:14.967700    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podStartSLOduration=1.96768707 podStartE2EDuration="1.96768707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:14.965652518 +0000 UTC m=+897.068406489" watchObservedRunningTime="2026-10-30 00:54:14.96768707 +0000 UTC m=+897.070441041"
Oct 30 00:54:15.586184 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:15.586146    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:23.586401 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:23.586363    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:23.617202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:23.616819    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:23.987147 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:23.987053    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:54:27.909880 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:54:27.909797    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:54:30.655973 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.655931    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"]
Oct 30 00:54:30.661749 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.661721    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.677652 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.677623    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"]
Oct 30 00:54:30.688316 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.688232    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-b565ec-openshift-default-dockercfg-sl86j\""
Oct 30 00:54:30.726222 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726189    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vsqv\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-kube-api-access-5vsqv\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726417 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726272    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726417 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726303    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726417 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726346    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726417 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726402    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b4fc90-89b9-427a-8c73-e770d836d92f-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726621 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726429    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726621 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726456    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726621 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726585    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.726771 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.726641    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.827497 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.827464    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.827758 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.827732    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.827844 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.827788    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.827895 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.827877    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vsqv\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-kube-api-access-5vsqv\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.827950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.827937    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828004 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.827980    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828058 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828034    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828058 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828032    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828155 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828056    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b4fc90-89b9-427a-8c73-e770d836d92f-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828155 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828098    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828447 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828419    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828663 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828644    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828755 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828734    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.828995 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.828977    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b4fc90-89b9-427a-8c73-e770d836d92f-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.830535 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.830514    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.830800 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.830781    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.840255 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.840230    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"]
Oct 30 00:54:30.844692 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.844675    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.850436 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.850411    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.850547 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.850526    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vsqv\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-kube-api-access-5vsqv\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:30.861027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.860987    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-da76a2-openshift-default-dockercfg-8l8nk\""
Oct 30 00:54:30.865208 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.865056    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"]
Oct 30 00:54:30.929163 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929082    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929163 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929128    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929380 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929266    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929380 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929319    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929380 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929338    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929477 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929426    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929553 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929539    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929607 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929594    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h46vk\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-kube-api-access-h46vk\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.929673 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.929660    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:30.977405 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:30.977371    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:31.032207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.030974    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.031072    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032503 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032286    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032503 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032340    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032503 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032465    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032673 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032550    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h46vk\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-kube-api-access-h46vk\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032673 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032587    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032673 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032621    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.032673 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032657    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.033156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.032739    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.033156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.033035    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.033156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.033035    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.034004 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.033312    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.034004 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.033538    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.034865 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.034844    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.036247 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.036227    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.055451 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.055418    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.057993 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.057860    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h46vk\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-kube-api-access-h46vk\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.183490 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.183400    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:31.293933 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.293902    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"]
Oct 30 00:54:31.294448 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:54:31.294424    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b4fc90_89b9_427a_8c73_e770d836d92f.slice/crio-18ac625ec18df232f649555a80b002f9068338e89a581c9008950289cae2948b WatchSource:0}: Error finding container 18ac625ec18df232f649555a80b002f9068338e89a581c9008950289cae2948b: Status 404 returned error can't find the container with id 18ac625ec18df232f649555a80b002f9068338e89a581c9008950289cae2948b
Oct 30 00:54:31.297113 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.297077    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:31.297232 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.297188    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:31.297232 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.297223    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:31.489793 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:54:31.489761    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c23c6e_0ea1_4786_bb03_b1c8b4731830.slice/crio-f8c61fac54a8114288a206405b489d87779b8a7f2da66245966083f2b40f1c24 WatchSource:0}: Error finding container f8c61fac54a8114288a206405b489d87779b8a7f2da66245966083f2b40f1c24: Status 404 returned error can't find the container with id f8c61fac54a8114288a206405b489d87779b8a7f2da66245966083f2b40f1c24
Oct 30 00:54:31.491635 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.491609    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"]
Oct 30 00:54:31.492687 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.492646    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:31.492764 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.492737    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:31.492814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.492788    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:31.858933 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.858894    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"]
Oct 30 00:54:31.864331 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.864307    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.882266 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.882233    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"]
Oct 30 00:54:31.944615 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.944576    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6jgx\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-kube-api-access-k6jgx\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.944806 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.944743    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.944964 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.944949    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.945072 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.945054    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.945129 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.945083    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.945237 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.945219    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a97b2579-278d-4aa6-b930-9172b08163a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.945286 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.945250    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a97b2579-278d-4aa6-b930-9172b08163a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.945321 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.945313    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:31.945361 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:31.945347    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.034253 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.034214    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" event={"ID":"32b4fc90-89b9-427a-8c73-e770d836d92f","Type":"ContainerStarted","Data":"c5e9f664fae4b62025a9048022977dbf18cd6b45fe42262525cd4bce19f5b92f"}
Oct 30 00:54:32.034456 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.034293    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" event={"ID":"32b4fc90-89b9-427a-8c73-e770d836d92f","Type":"ContainerStarted","Data":"18ac625ec18df232f649555a80b002f9068338e89a581c9008950289cae2948b"}
Oct 30 00:54:32.036105 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.036073    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" event={"ID":"66c23c6e-0ea1-4786-bb03-b1c8b4731830","Type":"ContainerStarted","Data":"17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7"}
Oct 30 00:54:32.036258 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.036126    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" event={"ID":"66c23c6e-0ea1-4786-bb03-b1c8b4731830","Type":"ContainerStarted","Data":"f8c61fac54a8114288a206405b489d87779b8a7f2da66245966083f2b40f1c24"}
Oct 30 00:54:32.047015 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.046979    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047032    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047053    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047074    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a97b2579-278d-4aa6-b930-9172b08163a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047094    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a97b2579-278d-4aa6-b930-9172b08163a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047120    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047154    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047572 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047539    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047637 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047612    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6jgx\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-kube-api-access-k6jgx\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047711 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047693    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047772 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047611    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.047891 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.047868    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a97b2579-278d-4aa6-b930-9172b08163a2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.048035 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.048007    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.048163 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.048065    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.050099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.050072    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.050258 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.050135    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a97b2579-278d-4aa6-b930-9172b08163a2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.071129 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.070477    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.071129 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.070813    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6jgx\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-kube-api-access-k6jgx\") pod \"maas-default-gateway-openshift-default-6db586dd6c-wl67t\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.095491 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.095428    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podStartSLOduration=2.095409179 podStartE2EDuration="2.095409179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:32.094601897 +0000 UTC m=+914.197355869" watchObservedRunningTime="2026-10-30 00:54:32.095409179 +0000 UTC m=+914.198163152"
Oct 30 00:54:32.138672 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.138572    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podStartSLOduration=2.138558231 podStartE2EDuration="2.138558231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:32.136643646 +0000 UTC m=+914.239397619" watchObservedRunningTime="2026-10-30 00:54:32.138558231 +0000 UTC m=+914.241312201"
Oct 30 00:54:32.187699 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.187579    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:32.188491 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.188447    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:32.189712 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.189448    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 00:54:32.189712 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.189502    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 00:54:32.595687 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:54:32.595653    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97b2579_278d_4aa6_b930_9172b08163a2.slice/crio-c9e6a1894fc8bbd77b1088bc551e32bcf7f22e8d25f7f6d47a50fec324f844db WatchSource:0}: Error finding container c9e6a1894fc8bbd77b1088bc551e32bcf7f22e8d25f7f6d47a50fec324f844db: Status 404 returned error can't find the container with id c9e6a1894fc8bbd77b1088bc551e32bcf7f22e8d25f7f6d47a50fec324f844db
Oct 30 00:54:32.597749 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.597723    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"]
Oct 30 00:54:32.597926 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.597898    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:32.597970 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.597961    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:32.598004 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.597993    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:32.983337 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.983205    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:32.985349 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.985321    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 00:54:32.985546 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:32.985518    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 00:54:33.082239 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.077606    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" event={"ID":"a97b2579-278d-4aa6-b930-9172b08163a2","Type":"ContainerStarted","Data":"2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f"}
Oct 30 00:54:33.082239 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.077674    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" event={"ID":"a97b2579-278d-4aa6-b930-9172b08163a2","Type":"ContainerStarted","Data":"c9e6a1894fc8bbd77b1088bc551e32bcf7f22e8d25f7f6d47a50fec324f844db"}
Oct 30 00:54:33.125718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.125668    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podStartSLOduration=2.125653827 podStartE2EDuration="2.125653827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:33.122032386 +0000 UTC m=+915.224786356" watchObservedRunningTime="2026-10-30 00:54:33.125653827 +0000 UTC m=+915.228407797"
Oct 30 00:54:33.192735 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.188631    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 00:54:33.192735 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.188693    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 00:54:33.192735 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.189211    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:33.192735 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.190912    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:33.192735 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.190958    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:33.978730 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.978690    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 00:54:33.979023 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:33.978991    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 00:54:34.184212 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:34.184027    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 00:54:34.184212 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:34.184099    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 00:54:34.200204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:34.196716    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:34.200204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:34.196784    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:35.002236 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:34.999650    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 00:54:35.002236 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:34.999729    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 00:54:35.191206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:35.190784    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 00:54:35.191206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:35.190869    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 00:54:35.191777 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:35.191510    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:35.191777 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:35.191572    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:35.991285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:35.990584    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 00:54:35.991285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:35.990648    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 00:54:36.187665 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:36.187621    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 00:54:36.187948 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:36.187919    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 00:54:36.191615 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:36.191581    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:36.192228 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:36.192195    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:36.978200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:36.978087    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 00:54:36.978200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:36.978167    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 00:54:37.192310 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:37.191824    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:37.192310 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:37.191885    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:37.192882 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:37.192334    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 00:54:37.192882 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:37.192404    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 00:54:37.978323 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:37.978282    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 00:54:37.978532 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:37.978338    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 00:54:38.184654 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:38.184612    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 00:54:38.184870 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:38.184675    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 00:54:38.188978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:38.188952    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:38.189132 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:38.188990    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:39.189261 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.189221    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:39.189757 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.189284    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:39.494251 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.493836    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:39.495582 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.495514    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:39.564213 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.563764    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 00:54:39.803372 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.803341    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:39.804050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.804027    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:39.846121 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:39.846090    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 00:54:40.189269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:40.189158    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused" start-of-body=
Oct 30 00:54:40.189269 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:40.189226    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": dial tcp 10.134.0.75:15021: connect: connection refused"
Oct 30 00:54:41.673129 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:41.673085    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:41.673665 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:41.673615    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:41.674671 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:41.674649    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 00:54:45.530885 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.530849    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"]
Oct 30 00:54:45.535358 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.535338    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.556043 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.556012    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"]
Oct 30 00:54:45.565615 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.565584    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-3c2596-openshift-default-dockercfg-mts5l\""
Oct 30 00:54:45.676621 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676590    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.676621 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676625    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.676904 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676646    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.676904 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676666    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.676904 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676789    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kngcg\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-kube-api-access-kngcg\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.676904 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676836    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.677032 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676941    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.677032 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.676980    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.677106 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.677093    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778075 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778030    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778133    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778285 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778269    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778362 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778286    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778362 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778305    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778362 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778328    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778362 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778355    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kngcg\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-kube-api-access-kngcg\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778561 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778379    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778561 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778449    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778663 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778595    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778721 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778701    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778854 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778827    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.778992 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.778972    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.779047 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.779016    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.781068 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.780999    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.781362 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.781300    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.798731 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.798700    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.798876 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.798857    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kngcg\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-kube-api-access-kngcg\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:45.850930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:45.850896    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:46.166287 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:46.166255    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"]
Oct 30 00:54:46.167300 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:54:46.167275    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e1d9ee_b865_4b21_a0ac_3f73bc0c6ba1.slice/crio-0143cf01cae0003bd6ad3b433c935779ae5b357d72d35658373d7724d2a0d519 WatchSource:0}: Error finding container 0143cf01cae0003bd6ad3b433c935779ae5b357d72d35658373d7724d2a0d519: Status 404 returned error can't find the container with id 0143cf01cae0003bd6ad3b433c935779ae5b357d72d35658373d7724d2a0d519
Oct 30 00:54:46.173599 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:46.173563    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:46.173690 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:46.173644    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:46.173690 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:46.173677    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:54:46.247681 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:46.247649    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" event={"ID":"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1","Type":"ContainerStarted","Data":"0143cf01cae0003bd6ad3b433c935779ae5b357d72d35658373d7724d2a0d519"}
Oct 30 00:54:47.270096 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:47.270060    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" event={"ID":"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1","Type":"ContainerStarted","Data":"201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e"}
Oct 30 00:54:47.861200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:47.857899    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:47.861200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:47.858033    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 00:54:47.861200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:47.858076    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 00:54:48.865411 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:48.865363    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 00:54:48.865901 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:48.865466    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 00:54:49.856200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:49.855620    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 00:54:49.856200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:49.855703    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 00:54:50.859207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:50.853608    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 00:54:50.859207 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:50.853712    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 00:54:51.855636 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:51.855600    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 00:54:51.855949 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:51.855917    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 00:54:52.851572 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:52.851535    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused" start-of-body=
Oct 30 00:54:52.851951 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:52.851607    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.76:15021/healthz/ready\": dial tcp 10.134.0.76:15021: connect: connection refused"
Oct 30 00:54:53.855548 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:53.855516    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:53.856044 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:53.855910    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:53.856707 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:53.856689    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 00:54:53.913963 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:54:53.913909    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podStartSLOduration=8.913895566 podStartE2EDuration="8.913895566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:54:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:54:47.338919026 +0000 UTC m=+929.441672996" watchObservedRunningTime="2026-10-30 00:54:53.913895566 +0000 UTC m=+936.016649537"
Oct 30 00:54:57.910537 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:54:57.910502    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:55:03.049159 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.049113    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"]
Oct 30 00:55:03.054210 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.054166    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.067800 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.067772    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"]
Oct 30 00:55:03.077278 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.077235    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b565ec\"/\"kube-root-ca.crt\""
Oct 30 00:55:03.077474 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.077291    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b565ec\"/\"distinct-2-w4-b565ec-kserve-self-signed-certs\""
Oct 30 00:55:03.084360 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.084335    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b565ec\"/\"openshift-service-ca.crt\""
Oct 30 00:55:03.084492 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.084404    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b565ec\"/\"default-dockercfg-7hgn4\""
Oct 30 00:55:03.171814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.171783    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-model-cache\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.171814 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.171814    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-tmp-dir\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.172051 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.171973    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a57c40-db65-47d3-a2f3-da243d3caf5a-tls-certs\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.172051 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.171996    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2g4g\" (UniqueName: \"kubernetes.io/projected/88a57c40-db65-47d3-a2f3-da243d3caf5a-kube-api-access-z2g4g\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.172317 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.172294    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-dshm\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.172476 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.172453    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-home\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.274371 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.274318    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-dshm\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.274617 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.274424    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-home\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.274617 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.274490    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-model-cache\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.274617 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.274535    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-tmp-dir\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.274950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.274927    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-model-cache\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.274950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.274940    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-home\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.275089 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.274972    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-tmp-dir\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.275089 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.275019    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a57c40-db65-47d3-a2f3-da243d3caf5a-tls-certs\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.275089 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.275056    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2g4g\" (UniqueName: \"kubernetes.io/projected/88a57c40-db65-47d3-a2f3-da243d3caf5a-kube-api-access-z2g4g\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.283220 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.277563    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a57c40-db65-47d3-a2f3-da243d3caf5a-tls-certs\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.283220 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.277625    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-dshm\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.295384 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.295352    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2g4g\" (UniqueName: \"kubernetes.io/projected/88a57c40-db65-47d3-a2f3-da243d3caf5a-kube-api-access-z2g4g\") pod \"distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") " pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.372976 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.372895    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:03.674188 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:55:03.674136    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88a57c40_db65_47d3_a2f3_da243d3caf5a.slice/crio-23d6225d75ce69e3d0b485c1a637b001503a82c20e23b13e4be9d539af2e48ae WatchSource:0}: Error finding container 23d6225d75ce69e3d0b485c1a637b001503a82c20e23b13e4be9d539af2e48ae: Status 404 returned error can't find the container with id 23d6225d75ce69e3d0b485c1a637b001503a82c20e23b13e4be9d539af2e48ae
Oct 30 00:55:03.674825 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:03.674799    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"]
Oct 30 00:55:04.400254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:04.400217    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" event={"ID":"88a57c40-db65-47d3-a2f3-da243d3caf5a","Type":"ContainerStarted","Data":"e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091"}
Oct 30 00:55:04.400254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:04.400259    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" event={"ID":"88a57c40-db65-47d3-a2f3-da243d3caf5a","Type":"ContainerStarted","Data":"23d6225d75ce69e3d0b485c1a637b001503a82c20e23b13e4be9d539af2e48ae"}
Oct 30 00:55:04.443807 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:04.443755    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podStartSLOduration=1.4437378299999999 podStartE2EDuration="1.44373783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:04.440053962 +0000 UTC m=+946.542807932" watchObservedRunningTime="2026-10-30 00:55:04.44373783 +0000 UTC m=+946.546491802"
Oct 30 00:55:05.373386 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:05.373335    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:13.376205 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:13.375309    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:14.215206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:14.212621    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:14.234242 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:14.234208    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:55:15.972788 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:15.972746    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"]
Oct 30 00:55:15.977569 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:15.977545    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:15.997655 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:15.997625    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"]
Oct 30 00:55:16.005515 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.005481    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.005816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.005793    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b145eea6-be64-417d-8c27-104f040067ee-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.005962 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.005942    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.006008 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.005978    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.006115 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.006098    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b145eea6-be64-417d-8c27-104f040067ee-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.006248 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.006226    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52z2h\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-kube-api-access-52z2h\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.006380 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.006359    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.006417 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.006409    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.006508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.006446    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107379 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107341    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b145eea6-be64-417d-8c27-104f040067ee-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107568 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107404    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52z2h\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-kube-api-access-52z2h\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107568 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107498    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107568 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107534    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107596    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107637    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107745 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107700    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b145eea6-be64-417d-8c27-104f040067ee-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107893 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107812    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.107893 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107855    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.108000 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.107923    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.108059 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.108001    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.108226 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.108203    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b145eea6-be64-417d-8c27-104f040067ee-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.108295 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.108258    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.108355 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.108324    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.118433 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.118402    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.118758 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.118741    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b145eea6-be64-417d-8c27-104f040067ee-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.129741 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.129711    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.132707 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.132688    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52z2h\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-kube-api-access-52z2h\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.305884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.305840    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:16.619149 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.619122    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"]
Oct 30 00:55:16.620335 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:55:16.620311    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb145eea6_be64_417d_8c27_104f040067ee.slice/crio-ba545b892204ede91ce2bf2e9beada017d4c5840892b4f9957370d398da22dea WatchSource:0}: Error finding container ba545b892204ede91ce2bf2e9beada017d4c5840892b4f9957370d398da22dea: Status 404 returned error can't find the container with id ba545b892204ede91ce2bf2e9beada017d4c5840892b4f9957370d398da22dea
Oct 30 00:55:16.622685 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.622652    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:16.622752 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.622721    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:16.622793 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:16.622752    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:17.534789 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:17.534753    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" event={"ID":"b145eea6-be64-417d-8c27-104f040067ee","Type":"ContainerStarted","Data":"a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092"}
Oct 30 00:55:17.535208 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:17.534815    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" event={"ID":"b145eea6-be64-417d-8c27-104f040067ee","Type":"ContainerStarted","Data":"ba545b892204ede91ce2bf2e9beada017d4c5840892b4f9957370d398da22dea"}
Oct 30 00:55:17.613623 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:17.613575    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podStartSLOduration=2.613561008 podStartE2EDuration="2.613561008s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:17.600103286 +0000 UTC m=+959.702857257" watchObservedRunningTime="2026-10-30 00:55:17.613561008 +0000 UTC m=+959.716314980"
Oct 30 00:55:18.306146 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:18.306111    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:18.307296 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:18.307272    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:55:18.307423 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:18.307325    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:55:19.306232 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:19.306155    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:55:19.306715 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:19.306242    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:55:20.306338 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:20.306304    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:55:20.306754 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:20.306359    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:55:21.306950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:21.306914    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:55:21.307531 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:21.306973    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:55:22.307247 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:22.307166    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:55:22.307671 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:22.307269    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:55:23.307099 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:23.307060    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused" start-of-body=
Oct 30 00:55:23.307365 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:23.307115    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.78:15021/healthz/ready\": dial tcp 10.134.0.78:15021: connect: connection refused"
Oct 30 00:55:24.311022 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:24.310992    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:24.311505 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:24.311301    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:24.311943 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:24.311924    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 00:55:27.910308 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:55:27.910266    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:55:30.683755 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.683472    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"]
Oct 30 00:55:30.699431 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.699399    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"]
Oct 30 00:55:30.699613 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.699528    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724425    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf03273-5153-4145-907a-1495fadd7036-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724484    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf03273-5153-4145-907a-1495fadd7036-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724740 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724582    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724740 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724611    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724740 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724674    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724740 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724719    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724747    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724776    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhlzc\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-kube-api-access-jhlzc\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.724978 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.724818    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.825884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.825850    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.825899    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhlzc\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-kube-api-access-jhlzc\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.825939    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.825996    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf03273-5153-4145-907a-1495fadd7036-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826050 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826020    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf03273-5153-4145-907a-1495fadd7036-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826314 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826079    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826314 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826103    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826314 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826205    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826314 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826276    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826590 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826549    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826674 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826629    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826740 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826717    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.826893 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.826867    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.827209 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.827162    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf03273-5153-4145-907a-1495fadd7036-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.828500 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.828476    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.829194 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.829158    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf03273-5153-4145-907a-1495fadd7036-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.848622 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.848597    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhlzc\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-kube-api-access-jhlzc\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.850330 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.850302    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:30.862405 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.862378    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"]
Oct 30 00:55:30.868488 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.868458    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.915966 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.915938    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"]
Oct 30 00:55:30.927950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.927920    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928132 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.927969    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928132 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.928064    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/128966aa-cede-48de-a21d-6b0b559ed75d-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928132 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.928097    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928132 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.928119    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928390 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.928138    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928390 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.928247    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/128966aa-cede-48de-a21d-6b0b559ed75d-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928390 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.928268    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5rzl\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-kube-api-access-k5rzl\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:30.928554 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:30.928509    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.021891 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.021809    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:31.029513 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029482    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.029654 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029526    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/128966aa-cede-48de-a21d-6b0b559ed75d-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.029654 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029546    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.029654 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029564    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.029654 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029588    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.029654 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029642    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/128966aa-cede-48de-a21d-6b0b559ed75d-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.029930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029672    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5rzl\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-kube-api-access-k5rzl\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.029930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029865    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.030036 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.029972    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.030317 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.030291    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.030317 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.030305    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/128966aa-cede-48de-a21d-6b0b559ed75d-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.030498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.030331    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.030498 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.030471    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.030768 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.030740    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.032600 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.032573    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.033015 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.032991    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/128966aa-cede-48de-a21d-6b0b559ed75d-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.050701 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.050666    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.051638 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.051573    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5rzl\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-kube-api-access-k5rzl\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.227639 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.226443    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:31.353649 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.353458    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"]
Oct 30 00:55:31.353649 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:55:31.353597    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaf03273_5153_4145_907a_1495fadd7036.slice/crio-bf157d97515a5392e965cc03355164d032f336f9958a4b57f7fd6ce9f6c5322f WatchSource:0}: Error finding container bf157d97515a5392e965cc03355164d032f336f9958a4b57f7fd6ce9f6c5322f: Status 404 returned error can't find the container with id bf157d97515a5392e965cc03355164d032f336f9958a4b57f7fd6ce9f6c5322f
Oct 30 00:55:31.357464 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.357411    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:31.358238 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.357529    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:31.358238 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.357600    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:31.552599 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:55:31.552560    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod128966aa_cede_48de_a21d_6b0b559ed75d.slice/crio-a2aa41e7c7cab3e9246d6a902f583233e0425845b364c13975603ba9008b52ea WatchSource:0}: Error finding container a2aa41e7c7cab3e9246d6a902f583233e0425845b364c13975603ba9008b52ea: Status 404 returned error can't find the container with id a2aa41e7c7cab3e9246d6a902f583233e0425845b364c13975603ba9008b52ea
Oct 30 00:55:31.554947 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.554908    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"]
Oct 30 00:55:31.555363 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.555331    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:31.555416 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.555398    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:31.555456 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.555425    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:31.656233 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.656195    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" event={"ID":"caf03273-5153-4145-907a-1495fadd7036","Type":"ContainerStarted","Data":"aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201"}
Oct 30 00:55:31.656429 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.656265    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" event={"ID":"caf03273-5153-4145-907a-1495fadd7036","Type":"ContainerStarted","Data":"bf157d97515a5392e965cc03355164d032f336f9958a4b57f7fd6ce9f6c5322f"}
Oct 30 00:55:31.659335 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.659295    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" event={"ID":"128966aa-cede-48de-a21d-6b0b559ed75d","Type":"ContainerStarted","Data":"960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c"}
Oct 30 00:55:31.659488 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.659355    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" event={"ID":"128966aa-cede-48de-a21d-6b0b559ed75d","Type":"ContainerStarted","Data":"a2aa41e7c7cab3e9246d6a902f583233e0425845b364c13975603ba9008b52ea"}
Oct 30 00:55:31.702641 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.702546    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podStartSLOduration=1.702525922 podStartE2EDuration="1.702525922s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:31.699514457 +0000 UTC m=+973.802268456" watchObservedRunningTime="2026-10-30 00:55:31.702525922 +0000 UTC m=+973.805279895"
Oct 30 00:55:31.751815 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:31.746122    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podStartSLOduration=1.746104376 podStartE2EDuration="1.746104376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:31.745694463 +0000 UTC m=+973.848448437" watchObservedRunningTime="2026-10-30 00:55:31.746104376 +0000 UTC m=+973.848858351"
Oct 30 00:55:32.040213 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:32.039448    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:32.040213 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:32.039573    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:55:32.040213 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:32.039623    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:55:32.230871 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:32.230529    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:32.230871 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:32.230632    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 00:55:32.230871 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:32.230666    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 00:55:33.023352 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:33.023321    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:55:33.023826 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:33.023801    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:55:33.227254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:33.227218    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 00:55:33.227433 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:33.227280    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 00:55:34.022668 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:34.022636    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:55:34.022837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:34.022685    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:55:34.227377 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:34.227345    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 00:55:34.227773 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:34.227406    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 00:55:35.022899 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:35.022869    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:55:35.023139 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:35.023118    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:55:35.227149 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:35.227118    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 00:55:35.227337 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:35.227204    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 00:55:36.022557 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:36.022526    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:55:36.023002 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:36.022576    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:55:36.227463 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:36.227428    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 00:55:36.227718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:36.227691    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 00:55:37.022448 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:37.022418    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:55:37.022686 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:37.022656    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:55:37.226855 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:37.226812    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 00:55:37.227060 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:37.226863    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 00:55:38.029339 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:38.029301    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused" start-of-body=
Oct 30 00:55:38.029948 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:38.029914    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.79:15021/healthz/ready\": dial tcp 10.134.0.79:15021: connect: connection refused"
Oct 30 00:55:38.227774 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:38.227738    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 00:55:38.227959 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:38.227804    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 00:55:39.096896 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:39.096867    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:39.098000 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:39.097979    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:39.101328 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:39.101302    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 00:55:39.451620 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:39.451541    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:39.453135 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:39.453112    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:39.454017 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:39.453997    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 00:55:45.940202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:45.940114    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"]
Oct 30 00:55:46.462514 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.462467    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"]
Oct 30 00:55:46.498201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.467395    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.546925    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.546983    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnthv\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-kube-api-access-vnthv\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.547046    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.547282    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df60db82-e952-4656-a298-6c0cf71e967d-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.547343    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.547386    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.547432    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df60db82-e952-4656-a298-6c0cf71e967d-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.547531    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.550466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.547603    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.653088 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.653048    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.653417 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.653389    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.653609 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.653589    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnthv\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-kube-api-access-vnthv\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.653808 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.653770    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.664295 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.664249    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df60db82-e952-4656-a298-6c0cf71e967d-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.664580 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.664562    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.664744 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.664728    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.674161 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.674133    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df60db82-e952-4656-a298-6c0cf71e967d-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.723561 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.674053    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-data\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.723804 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.686291    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df60db82-e952-4656-a298-6c0cf71e967d-istiod-ca-cert\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.723936 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.673793    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.724347 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.724300    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.724681 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.724654    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-certs\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.737242 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.725104    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-credential-socket\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.820254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.817426    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnthv\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-kube-api-access-vnthv\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.824612 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.824566    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df60db82-e952-4656-a298-6c0cf71e967d-istio-podinfo\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.845407 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.845369    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-envoy\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:46.922189 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:46.922140    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-istio-token\") pod \"e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") " pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:47.150807 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.150775    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:47.515317 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.515272    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"]
Oct 30 00:55:47.515512 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:55:47.515488    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf60db82_e952_4656_a298_6c0cf71e967d.slice/crio-4446bb13d5b06843316b7c1531fde4a7377ea20f1b129ba15a6ecfc79dbb8283 WatchSource:0}: Error finding container 4446bb13d5b06843316b7c1531fde4a7377ea20f1b129ba15a6ecfc79dbb8283: Status 404 returned error can't find the container with id 4446bb13d5b06843316b7c1531fde4a7377ea20f1b129ba15a6ecfc79dbb8283
Oct 30 00:55:47.518890 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.518848    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:47.518994 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.518946    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:47.519066 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.518992    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:55:47.931324 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.931284    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" event={"ID":"df60db82-e952-4656-a298-6c0cf71e967d","Type":"ContainerStarted","Data":"d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed"}
Oct 30 00:55:47.931527 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.931344    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" event={"ID":"df60db82-e952-4656-a298-6c0cf71e967d","Type":"ContainerStarted","Data":"4446bb13d5b06843316b7c1531fde4a7377ea20f1b129ba15a6ecfc79dbb8283"}
Oct 30 00:55:47.989736 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:47.989682    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podStartSLOduration=2.989666171 podStartE2EDuration="2.989666171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:55:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:55:47.987017655 +0000 UTC m=+990.089771640" watchObservedRunningTime="2026-10-30 00:55:47.989666171 +0000 UTC m=+990.092420212"
Oct 30 00:55:48.151360 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:48.151325    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:48.152782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:48.152753    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 00:55:48.152916 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:48.152813    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 00:55:49.158203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:49.157683    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 00:55:49.158203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:49.157754    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 00:55:50.162614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:50.162568    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 00:55:50.163111 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:50.162641    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 00:55:51.164202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:51.158619    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 00:55:51.164202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:51.158701    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 00:55:52.163210 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:52.162637    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 00:55:52.163210 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:52.162700    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 00:55:53.164980 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:53.164747    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 00:55:53.164980 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:53.164834    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 00:55:54.151966 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:54.151933    6011 patch_prober.go:29] interesting pod/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 00:55:54.152137 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:54.151994    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 00:55:55.258977 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:55.258940    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:55.259863 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:55.259802    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:55.260776 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:55:55.260758    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 00:55:57.910320 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:55:57.910285    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:56:09.029461 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:09.029387    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:27.986747 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:27.986693    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:28.038520 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:56:28.038477    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:56:28.055452 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:28.055408    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Oct 30 00:56:28.229458 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:28.228295    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 00:56:29.057202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:29.056278    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:29.057202 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:29.057016    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:56:29.391258 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:29.389700    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:29.418217 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:29.414335    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:56:29.889249 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:29.889215    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:56:29.913511 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:29.913482    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:56:45.867131 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.867092    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"]
Oct 30 00:56:45.871603 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.871576    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.889206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.889153    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"]
Oct 30 00:56:45.889731 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.889684    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.889907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.889736    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.889907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.889783    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900090ff-d224-4e19-ad98-9d9d0065ff7b-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.889907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.889837    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.890464 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.889915    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.890464 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.889989    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.890464 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.890150    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6pcc\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-kube-api-access-q6pcc\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.890464 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.890254    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.890464 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.890374    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.991945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.991904    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.991945 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.991945    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992247 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.991980    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900090ff-d224-4e19-ad98-9d9d0065ff7b-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992247 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992026    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992247 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992094    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992423 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992250    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992423 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992360    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6pcc\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-kube-api-access-q6pcc\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992423 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992412    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992451    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992481    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-certs\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992533    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-data\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992572    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-credential-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992843 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992821    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-socket\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.992967 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.992945    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900090ff-d224-4e19-ad98-9d9d0065ff7b-istiod-ca-cert\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.994465 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.994444    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-envoy\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:45.994651 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:45.994634    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-podinfo\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:46.012774 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.012747    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-token\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:46.012968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.012944    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6pcc\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-kube-api-access-q6pcc\") pod \"e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") " pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:46.191467 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.191372    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:46.210158 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.210121    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"]
Oct 30 00:56:46.216150 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.216126    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.244201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.242913    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"]
Oct 30 00:56:46.297884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.297457    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.297884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.297571    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f18fde74-276f-42ec-a7f7-dda2acb93287-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.297884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.297646    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.297884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.297700    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.297884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.297761    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.297884 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.297809    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.298410 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.298027    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.298410 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.298056    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz7lw\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-kube-api-access-tz7lw\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.298410 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.298144    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.399919 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.399887    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f18fde74-276f-42ec-a7f7-dda2acb93287-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400122 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.399969    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400122 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400011    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400122 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400063    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400122 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400101    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400412 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400213    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400412 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400240    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz7lw\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-kube-api-access-tz7lw\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400412 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400333    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400412 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400404    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400636 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400527    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400636 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400555    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-certs\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400748 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400707    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f18fde74-276f-42ec-a7f7-dda2acb93287-istiod-ca-cert\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400856 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400780    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-data\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.400856 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.400792    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-credential-socket\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.402643 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.402627    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-envoy\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.402786 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.402770    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-podinfo\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.420200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.420148    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-token\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.420377 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.420358    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz7lw\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-kube-api-access-tz7lw\") pod \"e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") " pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.523975 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:56:46.523938    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod900090ff_d224_4e19_ad98_9d9d0065ff7b.slice/crio-16033dc0f53ce584983102ebd59012a393cbce2f6bc91a93a1790439c78e776d WatchSource:0}: Error finding container 16033dc0f53ce584983102ebd59012a393cbce2f6bc91a93a1790439c78e776d: Status 404 returned error can't find the container with id 16033dc0f53ce584983102ebd59012a393cbce2f6bc91a93a1790439c78e776d
Oct 30 00:56:46.525063 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.525037    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"]
Oct 30 00:56:46.526533 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.526497    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:56:46.526646 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.526578    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:56:46.526646 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.526624    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:56:46.555893 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.555865    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:46.711048 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.711011    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" event={"ID":"900090ff-d224-4e19-ad98-9d9d0065ff7b","Type":"ContainerStarted","Data":"ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3"}
Oct 30 00:56:46.711310 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.711061    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" event={"ID":"900090ff-d224-4e19-ad98-9d9d0065ff7b","Type":"ContainerStarted","Data":"16033dc0f53ce584983102ebd59012a393cbce2f6bc91a93a1790439c78e776d"}
Oct 30 00:56:46.754980 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.754922    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podStartSLOduration=1.754907658 podStartE2EDuration="1.754907658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:56:46.753336057 +0000 UTC m=+1048.856090028" watchObservedRunningTime="2026-10-30 00:56:46.754907658 +0000 UTC m=+1048.857661629"
Oct 30 00:56:46.882237 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:56:46.882200    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf18fde74_276f_42ec_a7f7_dda2acb93287.slice/crio-b7cd043494d5c68fd98fba5a8bae57f512264bd8bdd2f886cb0e9818041117c0 WatchSource:0}: Error finding container b7cd043494d5c68fd98fba5a8bae57f512264bd8bdd2f886cb0e9818041117c0: Status 404 returned error can't find the container with id b7cd043494d5c68fd98fba5a8bae57f512264bd8bdd2f886cb0e9818041117c0
Oct 30 00:56:46.883462 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.883440    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"]
Oct 30 00:56:46.885351 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.885310    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:56:46.885506 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.885397    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:56:46.885506 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:46.885431    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:56:47.193049 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:47.192962    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:47.195038 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:47.194680    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:47.195038 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:47.194760    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:47.719834 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:47.719793    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" event={"ID":"f18fde74-276f-42ec-a7f7-dda2acb93287","Type":"ContainerStarted","Data":"6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8"}
Oct 30 00:56:47.720139 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:47.720115    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" event={"ID":"f18fde74-276f-42ec-a7f7-dda2acb93287","Type":"ContainerStarted","Data":"b7cd043494d5c68fd98fba5a8bae57f512264bd8bdd2f886cb0e9818041117c0"}
Oct 30 00:56:47.788256 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:47.787129    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podStartSLOduration=1.787105224 podStartE2EDuration="1.787105224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:56:47.7689836 +0000 UTC m=+1049.871737591" watchObservedRunningTime="2026-10-30 00:56:47.787105224 +0000 UTC m=+1049.889859196"
Oct 30 00:56:48.192992 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:48.192363    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:48.192992 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:48.192423    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:48.561782 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:48.561551    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:48.566326 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:48.566290    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:48.566604 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:48.566568    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:49.197201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:49.196765    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:49.197201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:49.196853    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:49.559206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:49.558868    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:49.559206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:49.558947    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:50.198201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:50.197736    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:50.198201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:50.197805    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:50.592395 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:50.592355    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:50.618348 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:50.618301    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:50.921718 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:50.921673    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:51.097521 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:50.930266    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:51.138203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:50.985711    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:51.194259 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:51.194052    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:51.194259 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:51.194137    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:51.548312 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:51.545637    6011 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:51.595367 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:51.594292    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:51.595367 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:51.594356    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:52.147216 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.127303    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:52.147633 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.147590    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:52.193201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.148954    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:52.224204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.199293    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:52.243542 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.224507    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:52.591392 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.591352    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:52.607341 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.607295    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:52.638205 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.629353    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:56:52.638205 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.629420    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:56:52.638205 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:52.629457    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:56:53.152390 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.152338    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:53.178223 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.166485    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:56:53.204199 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.185987    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:53.204199 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.191693    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:53.204199 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.191753    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:53.233518 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.233455    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:56:53.254495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.254412    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:56:53.560590 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.560555    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:53.560877 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:53.560849    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:54.192714 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:54.192674    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:54.193362 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:54.193312    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:54.570225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:54.569268    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:54.570225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:54.569330    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:55.191900 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:55.191860    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused" start-of-body=
Oct 30 00:56:55.192204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:55.191932    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": dial tcp 10.134.0.82:15021: connect: connection refused"
Oct 30 00:56:55.556755 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:55.556712    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 00:56:55.557335 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:55.556781    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 00:56:57.234345 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.234303    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:56:57.234806 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.234354    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:57.598434 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.598382    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:56:57.598816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.598783    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:56:57.782764 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.782732    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:57.783422 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.783391    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:57.786904 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.785960    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 00:56:57.928777 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.927574    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:57.928777 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.928660    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:57.942573 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:56:57.942462    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 00:56:57.948352 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:56:57.948286    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:57:06.188200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:06.187625    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 00:57:06.310201 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:06.309931    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 00:57:07.263466 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:07.263408    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": context deadline exceeded"
Oct 30 00:57:07.339113 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:07.339061    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 00:57:07.342404 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:07.342381    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:57:07.868737 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:07.868704    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:57:27.911074 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:57:27.910998    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:57:30.481487 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.481447    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8886-3e824c"]
Oct 30 00:57:30.487027 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.487008    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:30.500778 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.500750    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8886-3e824c"]
Oct 30 00:57:30.509369 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.509337    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-98k27\""
Oct 30 00:57:30.601468 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.601431    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9ht\" (UniqueName: \"kubernetes.io/projected/42953d26-45a1-4d67-a0da-7cd20856f998-kube-api-access-7q9ht\") pod \"test-curl-8886-3e824c\" (UID: \"42953d26-45a1-4d67-a0da-7cd20856f998\") " pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:30.702638 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.702599    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9ht\" (UniqueName: \"kubernetes.io/projected/42953d26-45a1-4d67-a0da-7cd20856f998-kube-api-access-7q9ht\") pod \"test-curl-8886-3e824c\" (UID: \"42953d26-45a1-4d67-a0da-7cd20856f998\") " pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:30.722744 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.722709    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9ht\" (UniqueName: \"kubernetes.io/projected/42953d26-45a1-4d67-a0da-7cd20856f998-kube-api-access-7q9ht\") pod \"test-curl-8886-3e824c\" (UID: \"42953d26-45a1-4d67-a0da-7cd20856f998\") " pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:30.802642 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:30.802597    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:31.113971 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:57:31.113941    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42953d26_45a1_4d67_a0da_7cd20856f998.slice/crio-2d4c3ab061dc381391f7c597c699519d311792b408d188017f38fdf260281de7 WatchSource:0}: Error finding container 2d4c3ab061dc381391f7c597c699519d311792b408d188017f38fdf260281de7: Status 404 returned error can't find the container with id 2d4c3ab061dc381391f7c597c699519d311792b408d188017f38fdf260281de7
Oct 30 00:57:31.114118 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:31.114017    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8886-3e824c"]
Oct 30 00:57:31.115783 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:31.115765    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 00:57:31.582384 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:31.582350    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8886-3e824c" event={"ID":"42953d26-45a1-4d67-a0da-7cd20856f998","Type":"ContainerStarted","Data":"2d4c3ab061dc381391f7c597c699519d311792b408d188017f38fdf260281de7"}
Oct 30 00:57:34.200559 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:34.200494    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:57:34.202878 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:34.202839    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:57:34.418203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:34.408745    6011 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/health\": context deadline exceeded"
Oct 30 00:57:34.600497 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:34.600436    6011 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/health\": context deadline exceeded"
Oct 30 00:57:34.883271 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:34.879260    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 00:57:35.292227 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:35.292159    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:57:35.293633 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:35.293585    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": context deadline exceeded"
Oct 30 00:57:35.309826 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:35.309782    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:57:35.310186 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:35.310151    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:57:35.883329 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:35.883286    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:57:35.884167 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:35.884132    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:57:36.265490 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:36.265404    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:57:36.265799 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:36.265780    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:57:36.265955 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:36.265941    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 00:57:38.257028 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:38.256947    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8886-3e824c" event={"ID":"42953d26-45a1-4d67-a0da-7cd20856f998","Type":"ContainerStarted","Data":"2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42"}
Oct 30 00:57:38.296729 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:38.296681    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8886-3e824c" podStartSLOduration=1.4981428399999999 podStartE2EDuration="8.296666858s" totalImagesPullingTime="6.798524018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:57:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:57:38.29428114 +0000 UTC m=+1100.397035112" watchObservedRunningTime="2026-10-30 00:57:38.296666858 +0000 UTC m=+1100.399420829"
Oct 30 00:57:39.129620 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.129581    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8886-3e824c"]
Oct 30 00:57:39.134867 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.134839    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8886-3e824c"]
Oct 30 00:57:39.300038 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.299994    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8886-e46353"]
Oct 30 00:57:39.300905 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.300882    6011 cpu_manager.go:414] "containerMap: removing container" podUID="42953d26-45a1-4d67-a0da-7cd20856f998" containerName="test-curl-8886-3e824c"
Oct 30 00:57:39.301009 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.300906    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42953d26-45a1-4d67-a0da-7cd20856f998" containerName="test-curl-8886-3e824c"
Oct 30 00:57:39.301113 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.301099    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="42953d26-45a1-4d67-a0da-7cd20856f998" containerName="test-curl-8886-3e824c"
Oct 30 00:57:39.305052 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.305030    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:39.317319 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.317294    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8886-e46353"]
Oct 30 00:57:39.405766 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.405680    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kdnv\" (UniqueName: \"kubernetes.io/projected/332031d9-cf14-48c6-a3a8-fd87bf1501a4-kube-api-access-4kdnv\") pod \"test-curl-8886-e46353\" (UID: \"332031d9-cf14-48c6-a3a8-fd87bf1501a4\") " pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:39.507554 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.507518    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kdnv\" (UniqueName: \"kubernetes.io/projected/332031d9-cf14-48c6-a3a8-fd87bf1501a4-kube-api-access-4kdnv\") pod \"test-curl-8886-e46353\" (UID: \"332031d9-cf14-48c6-a3a8-fd87bf1501a4\") " pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:39.527258 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.527228    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kdnv\" (UniqueName: \"kubernetes.io/projected/332031d9-cf14-48c6-a3a8-fd87bf1501a4-kube-api-access-4kdnv\") pod \"test-curl-8886-e46353\" (UID: \"332031d9-cf14-48c6-a3a8-fd87bf1501a4\") " pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:39.622339 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.622305    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:39.920902 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:57:39.920867    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod332031d9_cf14_48c6_a3a8_fd87bf1501a4.slice/crio-0d31f98d21f1fc38cee8bc4ca2acde2703af9c7dc766eddecf23f4acb6e84f34 WatchSource:0}: Error finding container 0d31f98d21f1fc38cee8bc4ca2acde2703af9c7dc766eddecf23f4acb6e84f34: Status 404 returned error can't find the container with id 0d31f98d21f1fc38cee8bc4ca2acde2703af9c7dc766eddecf23f4acb6e84f34
Oct 30 00:57:39.921243 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:39.921220    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8886-e46353"]
Oct 30 00:57:40.274749 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:40.274651    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8886-e46353" event={"ID":"332031d9-cf14-48c6-a3a8-fd87bf1501a4","Type":"ContainerStarted","Data":"53db2f56a3409516102849af9bd42eafc823d4bc47602cbe556d902d295d1777"}
Oct 30 00:57:40.274749 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:40.274704    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8886-e46353" event={"ID":"332031d9-cf14-48c6-a3a8-fd87bf1501a4","Type":"ContainerStarted","Data":"0d31f98d21f1fc38cee8bc4ca2acde2703af9c7dc766eddecf23f4acb6e84f34"}
Oct 30 00:57:40.274972 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:40.274793    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8886-3e824c" podUID="42953d26-45a1-4d67-a0da-7cd20856f998" containerName="test-curl-8886-3e824c" containerID="cri-o://2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42" gracePeriod=2
Oct 30 00:57:40.314011 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:40.313946    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8886-e46353" podStartSLOduration=1.313932183 podStartE2EDuration="1.313932183s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:57:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:57:40.313529058 +0000 UTC m=+1102.416283027" watchObservedRunningTime="2026-10-30 00:57:40.313932183 +0000 UTC m=+1102.416686145"
Oct 30 00:57:41.142877 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:41.142843    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8886-e46353"]
Oct 30 00:57:41.149234 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:41.149202    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8886-e46353"]
Oct 30 00:57:42.289419 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:42.289383    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8886-e46353" podUID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" containerName="test-curl-8886-e46353" containerID="cri-o://53db2f56a3409516102849af9bd42eafc823d4bc47602cbe556d902d295d1777" gracePeriod=2
Oct 30 00:57:42.494425 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:42.494398    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:42.546837 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:42.546805    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42953d26-45a1-4d67-a0da-7cd20856f998-kube-api-access-7q9ht\" (UniqueName: \"kubernetes.io/projected/42953d26-45a1-4d67-a0da-7cd20856f998-kube-api-access-7q9ht\") pod \"42953d26-45a1-4d67-a0da-7cd20856f998\" (UID: \"42953d26-45a1-4d67-a0da-7cd20856f998\") "
Oct 30 00:57:42.549069 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:42.549043    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42953d26-45a1-4d67-a0da-7cd20856f998-kube-api-access-7q9ht" pod "42953d26-45a1-4d67-a0da-7cd20856f998" (UID: "42953d26-45a1-4d67-a0da-7cd20856f998"). InnerVolumeSpecName "kube-api-access-7q9ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:57:42.648816 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:42.648777    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7q9ht\" (UniqueName: \"kubernetes.io/projected/42953d26-45a1-4d67-a0da-7cd20856f998-kube-api-access-7q9ht\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:57:43.302764 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:43.302729    6011 generic.go:356] "Generic (PLEG): container finished" podID="42953d26-45a1-4d67-a0da-7cd20856f998" containerID="2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42" exitCode=137
Oct 30 00:57:43.303336 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:43.302795    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:43.303336 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:43.302941    6011 scope.go:122] "RemoveContainer" containerID="2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42"
Oct 30 00:57:43.316465 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:43.316433    6011 scope.go:122] "RemoveContainer" containerID="2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42"
Oct 30 00:57:43.316785 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:57:43.316768    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42\": container with ID starting with 2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42 not found: ID does not exist" containerID="2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42"
Oct 30 00:57:43.316851 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:43.316794    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42"} err="failed to get container status \"2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42\": rpc error: code = NotFound desc = could not find container \"2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42\": container with ID starting with 2a0bfa9cc4dee6ad8e381e04d28da40875ee1a5ce506f70e57876e9f15605e42 not found: ID does not exist"
Oct 30 00:57:43.328245 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:57:43.328214    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8886-3e824c\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="42953d26-45a1-4d67-a0da-7cd20856f998" pod="opendatahub/test-curl-8886-3e824c"
Oct 30 00:57:44.021785 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:44.021753    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42953d26-45a1-4d67-a0da-7cd20856f998" path="/var/lib/kubelet/pods/42953d26-45a1-4d67-a0da-7cd20856f998/volumes"
Oct 30 00:57:44.312719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:44.312693    6011 generic.go:356] "Generic (PLEG): container finished" podID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" containerID="53db2f56a3409516102849af9bd42eafc823d4bc47602cbe556d902d295d1777" exitCode=137
Oct 30 00:57:44.470910 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:44.470885    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:44.581726 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:44.581637    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/332031d9-cf14-48c6-a3a8-fd87bf1501a4-kube-api-access-4kdnv\" (UniqueName: \"kubernetes.io/projected/332031d9-cf14-48c6-a3a8-fd87bf1501a4-kube-api-access-4kdnv\") pod \"332031d9-cf14-48c6-a3a8-fd87bf1501a4\" (UID: \"332031d9-cf14-48c6-a3a8-fd87bf1501a4\") "
Oct 30 00:57:44.583933 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:44.583901    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/332031d9-cf14-48c6-a3a8-fd87bf1501a4-kube-api-access-4kdnv" pod "332031d9-cf14-48c6-a3a8-fd87bf1501a4" (UID: "332031d9-cf14-48c6-a3a8-fd87bf1501a4"). InnerVolumeSpecName "kube-api-access-4kdnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:57:44.683150 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:44.683113    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kdnv\" (UniqueName: \"kubernetes.io/projected/332031d9-cf14-48c6-a3a8-fd87bf1501a4-kube-api-access-4kdnv\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:57:45.321855 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.321824    6011 scope.go:122] "RemoveContainer" containerID="53db2f56a3409516102849af9bd42eafc823d4bc47602cbe556d902d295d1777"
Oct 30 00:57:45.322321 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.321832    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:45.345230 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:57:45.345195    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8886-e46353\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:45.801479 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.801443    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"]
Oct 30 00:57:45.802162 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.802147    6011 cpu_manager.go:414] "containerMap: removing container" podUID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" containerName="test-curl-8886-e46353"
Oct 30 00:57:45.802231 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.802165    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" containerName="test-curl-8886-e46353"
Oct 30 00:57:45.802330 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.802319    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" containerName="test-curl-8886-e46353"
Oct 30 00:57:45.807374 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.807325    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.822047 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:57:45.822011    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8886-e46353\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" pod="opendatahub/test-curl-8886-e46353"
Oct 30 00:57:45.823068 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.823043    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"]
Oct 30 00:57:45.899562 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.899516    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-credential-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.899562 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.899567    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-envoy\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.899832 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.899648    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-data\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.899832 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.899768    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istiod-ca-cert\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.899908 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.899882    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcr4c\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-kube-api-access-jcr4c\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.899983 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.899963    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-certs\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.900020 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.900007    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-podinfo\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.900065 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.900053    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:45.900105 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:45.900073    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-token\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.000853 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.000818    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcr4c\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-kube-api-access-jcr4c\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001055 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.000885    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-certs\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001055 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.000929    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-podinfo\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001055 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.000992    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001055 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001017    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-token\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001308 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001088    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-credential-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001308 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001211    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-envoy\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001308 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001279    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-data\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001308 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001292    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-certs\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001530 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001373    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istiod-ca-cert\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001530 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001502    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001624 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001548    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-credential-socket\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001705 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001674    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-data\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.001982 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.001963    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istiod-ca-cert\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.004085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.004024    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-podinfo\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.004085 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.004036    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-envoy\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.021614 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.021584    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-token\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.021963 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.021940    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcr4c\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-kube-api-access-jcr4c\") pod \"e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") " pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.025867 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.025843    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="332031d9-cf14-48c6-a3a8-fd87bf1501a4" path="/var/lib/kubelet/pods/332031d9-cf14-48c6-a3a8-fd87bf1501a4/volumes"
Oct 30 00:57:46.125431 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.125346    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:46.445162 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.445128    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"]
Oct 30 00:57:46.445553 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:57:46.445379    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a1b0938_e3e2_4080_820e_4eddbeb2cb84.slice/crio-7d0d743d29ebd454c751d2650699ba4478781cd9f7f0d75f7c3377b9f2987d70 WatchSource:0}: Error finding container 7d0d743d29ebd454c751d2650699ba4478781cd9f7f0d75f7c3377b9f2987d70: Status 404 returned error can't find the container with id 7d0d743d29ebd454c751d2650699ba4478781cd9f7f0d75f7c3377b9f2987d70
Oct 30 00:57:46.447968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.447933    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:57:46.448075 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.448011    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:57:46.448075 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:46.448058    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:57:47.364396 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:47.364353    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" event={"ID":"5a1b0938-e3e2-4080-820e-4eddbeb2cb84","Type":"ContainerStarted","Data":"1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef"}
Oct 30 00:57:47.364737 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:47.364687    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" event={"ID":"5a1b0938-e3e2-4080-820e-4eddbeb2cb84","Type":"ContainerStarted","Data":"7d0d743d29ebd454c751d2650699ba4478781cd9f7f0d75f7c3377b9f2987d70"}
Oct 30 00:57:47.422113 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:47.422054    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" podStartSLOduration=2.422034582 podStartE2EDuration="2.422034582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:57:47.419871617 +0000 UTC m=+1109.522625587" watchObservedRunningTime="2026-10-30 00:57:47.422034582 +0000 UTC m=+1109.524788555"
Oct 30 00:57:48.125844 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:48.125807    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:48.127678 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:48.127651    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 00:57:48.127828 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:48.127707    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 00:57:49.125906 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:49.125870    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 00:57:49.126382 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:49.125918    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 00:57:50.125990 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:50.125948    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 00:57:50.126543 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:50.126022    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 00:57:51.126081 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:51.126040    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 00:57:51.126530 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:51.126098    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 00:57:52.126644 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:52.126603    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused" start-of-body=
Oct 30 00:57:52.127049 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:52.126661    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.86:15021/healthz/ready\": dial tcp 10.134.0.86:15021: connect: connection refused"
Oct 30 00:57:53.494584 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:53.494552    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:53.496666 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:53.496640    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:53.497970 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:53.497953    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 00:57:58.651210 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:57:58.650419    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:57:58.874044 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:58.874006    6011 patch_prober.go:29] interesting pod/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:57:58.894604 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:58.894559    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:57:58.994431 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:58.994324    6011 patch_prober.go:29] interesting pod/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:57:58.994757 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:58.994723    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:57:59.229915 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:59.229869    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:57:59.300502 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:59.281270    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": context deadline exceeded"
Oct 30 00:57:59.330379 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:57:59.329257    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:00.343403 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:00.343331    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:00.547836 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:00.547803    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:58:01.232327 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:01.232278    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:01.349823 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:01.349773    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" probeResult="failure" output="Get \"https://10.134.0.72:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:01.360247 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:01.350409    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:01.516263 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:01.478663    6011 prober.go:122] "Probe failed" probeType="Liveness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:02.267577 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:02.266843    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77" podUID="ffc56f52-428f-4e01-bf96-9a42df58b065" containerName="main" probeResult="failure" output="Get \"https://10.134.0.44:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:02.401589 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:02.401537    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.77:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:58:02.429906 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:02.429857    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:58:02.438160 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:02.438124    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77"
Oct 30 00:58:02.481313 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:02.480810    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 00:58:04.398808 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.398772    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"]
Oct 30 00:58:04.404254 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.404227    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:04.419290 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.419258    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"]
Oct 30 00:58:04.429112 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.429083    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-baf5781e\""
Oct 30 00:58:04.429286 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.429207    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-baf5781e-dockercfg-4c5wk\""
Oct 30 00:58:04.456270 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.456232    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfqzg\" (UniqueName: \"kubernetes.io/projected/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-kube-api-access-mfqzg\") pod \"payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") " pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:04.456719 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.456688    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-plugins-config-volume\") pod \"payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") " pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:04.558235 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.558195    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-plugins-config-volume\") pod \"payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") " pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:04.558448 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.558357    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfqzg\" (UniqueName: \"kubernetes.io/projected/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-kube-api-access-mfqzg\") pod \"payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") " pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:04.558955 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.558931    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-plugins-config-volume\") pod \"payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") " pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:04.578844 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.578817    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfqzg\" (UniqueName: \"kubernetes.io/projected/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-kube-api-access-mfqzg\") pod \"payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") " pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:04.723753 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:04.723662    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:05.033200 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:05.030933    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"]
Oct 30 00:58:05.290127 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:05.290088    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m" event={"ID":"cb5cddd4-aa32-4018-a2ee-81586f9e2f03","Type":"ContainerStarted","Data":"b39159431c53b545a0de5d7952b73e58e98dfbfecd2256f20bd7769ec79863f2"}
Oct 30 00:58:05.290320 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:05.290155    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m" event={"ID":"cb5cddd4-aa32-4018-a2ee-81586f9e2f03","Type":"ContainerStarted","Data":"1d8612a6f7e6ee1d8278201bf2bdb01cf173c5427f656d968e8a0e8a57b8b35c"}
Oct 30 00:58:05.290320 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:05.290215    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:05.330704 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:05.330646    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m" podStartSLOduration=1.330632469 podStartE2EDuration="1.330632469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:05.329008612 +0000 UTC m=+1127.431762583" watchObservedRunningTime="2026-10-30 00:58:05.330632469 +0000 UTC m=+1127.433386460"
Oct 30 00:58:15.609199 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:15.606088    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"]
Oct 30 00:58:16.422875 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.422826    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"]
Oct 30 00:58:16.423273 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.423238    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:58:16.465327 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.442855    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.599206 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.595959    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 00:58:16.620339 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.620305    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 00:58:16.620883 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.604071    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 00:58:16.625524 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.625498    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-lrqvm\""
Oct 30 00:58:16.666087 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.665964    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g87qw\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-kube-api-access-g87qw\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.666599 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.666576    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.685554 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.685474    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16e2564-d89c-4ecf-95fa-b41fe949f901-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.685744 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.685721    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.686092 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.686063    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.687345 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.687314    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.687515 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.687494    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.802204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.801146    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.802204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.801325    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.802204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.801355    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.802204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.801488    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g87qw\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-kube-api-access-g87qw\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.802204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.801578    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.802204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.801624    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16e2564-d89c-4ecf-95fa-b41fe949f901-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.802204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.801648    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.824204 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.802927    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.971963 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.971872    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g87qw\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-kube-api-access-g87qw\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.992840 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.992767    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.993484 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.993427    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.993997 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.993950    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:16.999535 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:16.999484    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16e2564-d89c-4ecf-95fa-b41fe949f901-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:17.000225 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:17.000201    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-nvwbj\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:17.136203 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:17.134587    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:17.593878 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:58:17.593847    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd16e2564_d89c_4ecf_95fa_b41fe949f901.slice/crio-98b7a6b1e70511d36252166709e52c35b8c0d9183c1887bf0870a4055b83a281 WatchSource:0}: Error finding container 98b7a6b1e70511d36252166709e52c35b8c0d9183c1887bf0870a4055b83a281: Status 404 returned error can't find the container with id 98b7a6b1e70511d36252166709e52c35b8c0d9183c1887bf0870a4055b83a281
Oct 30 00:58:17.595140 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:17.595114    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"]
Oct 30 00:58:18.566129 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:18.566087    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj" event={"ID":"d16e2564-d89c-4ecf-95fa-b41fe949f901","Type":"ContainerStarted","Data":"98b7a6b1e70511d36252166709e52c35b8c0d9183c1887bf0870a4055b83a281"}
Oct 30 00:58:20.456304 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:20.456268    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:58:20.456615 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:20.456334    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 00:58:20.585424 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:20.585374    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj" event={"ID":"d16e2564-d89c-4ecf-95fa-b41fe949f901","Type":"ContainerStarted","Data":"bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e"}
Oct 30 00:58:20.585619 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:20.585434    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:20.655926 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:20.655872    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj" podStartSLOduration=2.796912636 podStartE2EDuration="5.655858307s" totalImagesPullingTime="2.858945671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:20.65462511 +0000 UTC m=+1142.757379080" watchObservedRunningTime="2026-10-30 00:58:20.655858307 +0000 UTC m=+1142.758612277"
Oct 30 00:58:21.594462 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:21.594423    6011 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-nvwbj container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 00:58:21.594872 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:21.594487    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj" podUID="d16e2564-d89c-4ecf-95fa-b41fe949f901" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 00:58:24.594534 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:24.594496    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 00:58:27.911681 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:58:27.911641    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:58:51.129156 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.129125    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"]
Oct 30 00:58:51.132606 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.132585    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.147503 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.147472    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2497727d-39aa-4e52-ba43-cf53c799c0b2-openshift-service-ca\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.147661 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.147525    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.147661 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.147589    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-metrics-certs\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.147950 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.147812    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r78d\" (UniqueName: \"kubernetes.io/projected/2497727d-39aa-4e52-ba43-cf53c799c0b2-kube-api-access-6r78d\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.149751 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.149726    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"]
Oct 30 00:58:51.157972 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.157948    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5f146c27-serving-cert\""
Oct 30 00:58:51.249726 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.249684    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r78d\" (UniqueName: \"kubernetes.io/projected/2497727d-39aa-4e52-ba43-cf53c799c0b2-kube-api-access-6r78d\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.249930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.249818    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2497727d-39aa-4e52-ba43-cf53c799c0b2-openshift-service-ca\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.249930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.249854    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.249930 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.249909    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-metrics-certs\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.250111 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:58:51.250042    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-serving-cert: secret "maas-api-e2e-ait-5f146c27-serving-cert" not found
Oct 30 00:58:51.250196 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:58:51.250124    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls podName:2497727d-39aa-4e52-ba43-cf53c799c0b2 nodeName:}" failed. No retries permitted until 2026-10-30 00:58:51.750105544 +0000 UTC m=+1173.852859510 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls") pod "maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm" (UID: "2497727d-39aa-4e52-ba43-cf53c799c0b2") : secret "maas-api-e2e-ait-5f146c27-serving-cert" not found
Oct 30 00:58:51.250564 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.250544    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2497727d-39aa-4e52-ba43-cf53c799c0b2-openshift-service-ca\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.252680 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.252660    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-metrics-certs\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.270576 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.270551    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r78d\" (UniqueName: \"kubernetes.io/projected/2497727d-39aa-4e52-ba43-cf53c799c0b2-kube-api-access-6r78d\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.760010 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.759951    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:51.762765 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:51.762745    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls\") pod \"maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:52.050151 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:52.050110    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:52.363524 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:58:52.363492    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2497727d_39aa_4e52_ba43_cf53c799c0b2.slice/crio-7e993285abf772f38211451c323a35125da26dcfe77ebc53f5e1776c8db42442 WatchSource:0}: Error finding container 7e993285abf772f38211451c323a35125da26dcfe77ebc53f5e1776c8db42442: Status 404 returned error can't find the container with id 7e993285abf772f38211451c323a35125da26dcfe77ebc53f5e1776c8db42442
Oct 30 00:58:52.365495 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:52.365463    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"]
Oct 30 00:58:52.861692 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:52.861657    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm" event={"ID":"2497727d-39aa-4e52-ba43-cf53c799c0b2","Type":"ContainerStarted","Data":"6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8"}
Oct 30 00:58:52.861862 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:52.861708    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm" event={"ID":"2497727d-39aa-4e52-ba43-cf53c799c0b2","Type":"ContainerStarted","Data":"7e993285abf772f38211451c323a35125da26dcfe77ebc53f5e1776c8db42442"}
Oct 30 00:58:52.861862 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:52.861738    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:58:52.904008 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:52.903951    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm" podStartSLOduration=1.595360946 podStartE2EDuration="1.903936163s" totalImagesPullingTime="308.575217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:58:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:58:52.902279763 +0000 UTC m=+1175.005033731" watchObservedRunningTime="2026-10-30 00:58:52.903936163 +0000 UTC m=+1175.006690134"
Oct 30 00:58:57.802144 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:57.802103    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"]
Oct 30 00:58:57.916869 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:58:57.916830    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:58:58.877106 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:58:58.877060    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:59:02.803928 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:02.803851    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m" podUID="cb5cddd4-aa32-4018-a2ee-81586f9e2f03" containerName="payload-processing" containerID="cri-o://b39159431c53b545a0de5d7952b73e58e98dfbfecd2256f20bd7769ec79863f2" gracePeriod=25
Oct 30 00:59:02.944272 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:02.944238    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m" event={"ID":"cb5cddd4-aa32-4018-a2ee-81586f9e2f03","Type":"ContainerDied","Data":"b39159431c53b545a0de5d7952b73e58e98dfbfecd2256f20bd7769ec79863f2"}
Oct 30 00:59:02.944462 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:02.944161    6011 generic.go:356] "Generic (PLEG): container finished" podID="cb5cddd4-aa32-4018-a2ee-81586f9e2f03" containerID="b39159431c53b545a0de5d7952b73e58e98dfbfecd2256f20bd7769ec79863f2" exitCode=0
Oct 30 00:59:03.082348 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.082310    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:59:03.237303 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.237270    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-plugins-config-volume\") pod \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") "
Oct 30 00:59:03.237608 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.237584    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-kube-api-access-mfqzg\" (UniqueName: \"kubernetes.io/projected/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-kube-api-access-mfqzg\") pod \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\" (UID: \"cb5cddd4-aa32-4018-a2ee-81586f9e2f03\") "
Oct 30 00:59:03.237750 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.237728    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-plugins-config-volume" pod "cb5cddd4-aa32-4018-a2ee-81586f9e2f03" (UID: "cb5cddd4-aa32-4018-a2ee-81586f9e2f03"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:03.238105 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.238090    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:59:03.239883 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.239857    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-kube-api-access-mfqzg" pod "cb5cddd4-aa32-4018-a2ee-81586f9e2f03" (UID: "cb5cddd4-aa32-4018-a2ee-81586f9e2f03"). InnerVolumeSpecName "kube-api-access-mfqzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:03.339310 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.339236    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfqzg\" (UniqueName: \"kubernetes.io/projected/cb5cddd4-aa32-4018-a2ee-81586f9e2f03-kube-api-access-mfqzg\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:59:03.954259 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.954232    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"
Oct 30 00:59:03.954730 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.954229    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m" event={"ID":"cb5cddd4-aa32-4018-a2ee-81586f9e2f03","Type":"ContainerDied","Data":"1d8612a6f7e6ee1d8278201bf2bdb01cf173c5427f656d968e8a0e8a57b8b35c"}
Oct 30 00:59:03.954730 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:03.954365    6011 scope.go:122] "RemoveContainer" containerID="b39159431c53b545a0de5d7952b73e58e98dfbfecd2256f20bd7769ec79863f2"
Oct 30 00:59:04.009236 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:04.009204    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"]
Oct 30 00:59:04.022721 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:04.022694    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-baf5781e-8469c46d4-6vw6m"]
Oct 30 00:59:06.024783 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:06.024745    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb5cddd4-aa32-4018-a2ee-81586f9e2f03" path="/var/lib/kubelet/pods/cb5cddd4-aa32-4018-a2ee-81586f9e2f03/volumes"
Oct 30 00:59:26.069449 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.069405    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"]
Oct 30 00:59:26.070051 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.069630    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm" podUID="2497727d-39aa-4e52-ba43-cf53c799c0b2" containerName="maas-api" containerID="cri-o://6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8" gracePeriod=30
Oct 30 00:59:26.351211 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.351159    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:59:26.457865 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.457829    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2497727d-39aa-4e52-ba43-cf53c799c0b2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2497727d-39aa-4e52-ba43-cf53c799c0b2-openshift-service-ca\") pod \"2497727d-39aa-4e52-ba43-cf53c799c0b2\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") "
Oct 30 00:59:26.458041 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.457913    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-metrics-certs\") pod \"2497727d-39aa-4e52-ba43-cf53c799c0b2\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") "
Oct 30 00:59:26.458041 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.458019    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls\") pod \"2497727d-39aa-4e52-ba43-cf53c799c0b2\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") "
Oct 30 00:59:26.458164 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.458145    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2497727d-39aa-4e52-ba43-cf53c799c0b2-kube-api-access-6r78d\" (UniqueName: \"kubernetes.io/projected/2497727d-39aa-4e52-ba43-cf53c799c0b2-kube-api-access-6r78d\") pod \"2497727d-39aa-4e52-ba43-cf53c799c0b2\" (UID: \"2497727d-39aa-4e52-ba43-cf53c799c0b2\") "
Oct 30 00:59:26.458299 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.458269    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2497727d-39aa-4e52-ba43-cf53c799c0b2-openshift-service-ca" pod "2497727d-39aa-4e52-ba43-cf53c799c0b2" (UID: "2497727d-39aa-4e52-ba43-cf53c799c0b2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 00:59:26.458871 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.458846    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2497727d-39aa-4e52-ba43-cf53c799c0b2-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:59:26.460351 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.460323    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-metrics-certs" pod "2497727d-39aa-4e52-ba43-cf53c799c0b2" (UID: "2497727d-39aa-4e52-ba43-cf53c799c0b2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:26.460460 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.460363    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls" pod "2497727d-39aa-4e52-ba43-cf53c799c0b2" (UID: "2497727d-39aa-4e52-ba43-cf53c799c0b2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 00:59:26.460901 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.460879    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2497727d-39aa-4e52-ba43-cf53c799c0b2-kube-api-access-6r78d" pod "2497727d-39aa-4e52-ba43-cf53c799c0b2" (UID: "2497727d-39aa-4e52-ba43-cf53c799c0b2"). InnerVolumeSpecName "kube-api-access-6r78d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 00:59:26.569832 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.567238    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r78d\" (UniqueName: \"kubernetes.io/projected/2497727d-39aa-4e52-ba43-cf53c799c0b2-kube-api-access-6r78d\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:59:26.569832 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.569659    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:59:26.569832 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:26.569694    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2497727d-39aa-4e52-ba43-cf53c799c0b2-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 00:59:27.167995 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.167956    6011 generic.go:356] "Generic (PLEG): container finished" podID="2497727d-39aa-4e52-ba43-cf53c799c0b2" containerID="6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8" exitCode=0
Oct 30 00:59:27.168508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.168022    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"
Oct 30 00:59:27.168508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.168039    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm" event={"ID":"2497727d-39aa-4e52-ba43-cf53c799c0b2","Type":"ContainerDied","Data":"6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8"}
Oct 30 00:59:27.168508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.168096    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm" event={"ID":"2497727d-39aa-4e52-ba43-cf53c799c0b2","Type":"ContainerDied","Data":"7e993285abf772f38211451c323a35125da26dcfe77ebc53f5e1776c8db42442"}
Oct 30 00:59:27.168508 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.168122    6011 scope.go:122] "RemoveContainer" containerID="6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8"
Oct 30 00:59:27.181662 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.181637    6011 scope.go:122] "RemoveContainer" containerID="6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8"
Oct 30 00:59:27.182185 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:59:27.182139    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8\": container with ID starting with 6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8 not found: ID does not exist" containerID="6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8"
Oct 30 00:59:27.182286 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.182200    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8"} err="failed to get container status \"6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8\": rpc error: code = NotFound desc = could not find container \"6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8\": container with ID starting with 6c8c31f7fd1b8640d4b85f881e338121c0dcde3f851f782d5293df6c7048f4b8 not found: ID does not exist"
Oct 30 00:59:27.227483 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.227451    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"]
Oct 30 00:59:27.240834 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:27.240805    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f146c27-64b765dc5d-lwgmm"]
Oct 30 00:59:27.918641 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:59:27.918601    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 00:59:28.022695 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:28.022658    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2497727d-39aa-4e52-ba43-cf53c799c0b2" path="/var/lib/kubelet/pods/2497727d-39aa-4e52-ba43-cf53c799c0b2/volumes"
Oct 30 00:59:35.344332 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.344292    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"]
Oct 30 00:59:35.345078 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.345061    6011 cpu_manager.go:414] "containerMap: removing container" podUID="cb5cddd4-aa32-4018-a2ee-81586f9e2f03" containerName="payload-processing"
Oct 30 00:59:35.345168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.345080    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb5cddd4-aa32-4018-a2ee-81586f9e2f03" containerName="payload-processing"
Oct 30 00:59:35.345168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.345100    6011 cpu_manager.go:414] "containerMap: removing container" podUID="2497727d-39aa-4e52-ba43-cf53c799c0b2" containerName="maas-api"
Oct 30 00:59:35.345168 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.345106    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2497727d-39aa-4e52-ba43-cf53c799c0b2" containerName="maas-api"
Oct 30 00:59:35.345300 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.345236    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="2497727d-39aa-4e52-ba43-cf53c799c0b2" containerName="maas-api"
Oct 30 00:59:35.345300 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.345256    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb5cddd4-aa32-4018-a2ee-81586f9e2f03" containerName="payload-processing"
Oct 30 00:59:35.348825 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.348809    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.365899 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.365862    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"]
Oct 30 00:59:35.375658 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.375626    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-95b31f79-serving-cert\""
Oct 30 00:59:35.519077 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.519041    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfzcb\" (UniqueName: \"kubernetes.io/projected/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-kube-api-access-pfzcb\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.519535 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.519508    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-metrics-certs\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.520907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.519574    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-maas-api-tls\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.520907 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.519729    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-openshift-service-ca\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.621663 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.621563    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-metrics-certs\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.621663 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.621633    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-maas-api-tls\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.621968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.621726    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-openshift-service-ca\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.621968 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.621942    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfzcb\" (UniqueName: \"kubernetes.io/projected/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-kube-api-access-pfzcb\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.622640 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.622557    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-openshift-service-ca\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.624708 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.624682    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-maas-api-tls\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.624911 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.624694    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-metrics-certs\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.642294 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.642265    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfzcb\" (UniqueName: \"kubernetes.io/projected/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-kube-api-access-pfzcb\") pod \"maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.664306 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.664279    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:35.971691 ip-10-0-131-254 kubenswrapper[6011]: W1030 00:59:35.971654    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf0d8869_b6b8_46ea_a4ad_1b327a66c031.slice/crio-6bc31f85d4ca2bd364567b056dc718d20a77c8cf2bdf6fb1b73c8e0ceb88ad5c WatchSource:0}: Error finding container 6bc31f85d4ca2bd364567b056dc718d20a77c8cf2bdf6fb1b73c8e0ceb88ad5c: Status 404 returned error can't find the container with id 6bc31f85d4ca2bd364567b056dc718d20a77c8cf2bdf6fb1b73c8e0ceb88ad5c
Oct 30 00:59:35.972399 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:35.972373    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"]
Oct 30 00:59:36.233833 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:36.233741    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c" event={"ID":"bf0d8869-b6b8-46ea-a4ad-1b327a66c031","Type":"ContainerStarted","Data":"6bc31f85d4ca2bd364567b056dc718d20a77c8cf2bdf6fb1b73c8e0ceb88ad5c"}
Oct 30 00:59:37.243258 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:37.243213    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c" event={"ID":"bf0d8869-b6b8-46ea-a4ad-1b327a66c031","Type":"ContainerStarted","Data":"01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb"}
Oct 30 00:59:37.243664 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:37.243475    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:37.286960 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:37.286906    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c" podStartSLOduration=1.973395428 podStartE2EDuration="2.286892012s" totalImagesPullingTime="313.496584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 00:59:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 00:59:37.284120574 +0000 UTC m=+1219.386874546" watchObservedRunningTime="2026-10-30 00:59:37.286892012 +0000 UTC m=+1219.389645982"
Oct 30 00:59:43.257231 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:43.257166    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 00:59:57.719415 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:57.719368    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-wl67t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 00:59:57.719952 ip-10-0-131-254 kubenswrapper[6011]: I1030 00:59:57.719459    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 00:59:57.911088 ip-10-0-131-254 kubenswrapper[6011]: E1030 00:59:57.911047    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:00:11.837035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:11.837000    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"]
Oct 30 01:00:11.837858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:11.837233    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c" podUID="bf0d8869-b6b8-46ea-a4ad-1b327a66c031" containerName="maas-api" containerID="cri-o://01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb" gracePeriod=30
Oct 30 01:00:12.149479 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.149444    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 01:00:12.235968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.235929    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-maas-api-tls\") pod \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") "
Oct 30 01:00:12.236200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.236046    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-metrics-certs\") pod \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") "
Oct 30 01:00:12.236200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.236167    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-kube-api-access-pfzcb\" (UniqueName: \"kubernetes.io/projected/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-kube-api-access-pfzcb\") pod \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") "
Oct 30 01:00:12.236336 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.236269    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-openshift-service-ca\") pod \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\" (UID: \"bf0d8869-b6b8-46ea-a4ad-1b327a66c031\") "
Oct 30 01:00:12.237243 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.236982    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-openshift-service-ca" pod "bf0d8869-b6b8-46ea-a4ad-1b327a66c031" (UID: "bf0d8869-b6b8-46ea-a4ad-1b327a66c031"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:00:12.237499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.237461    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:00:12.238793 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.238519    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-maas-api-tls" pod "bf0d8869-b6b8-46ea-a4ad-1b327a66c031" (UID: "bf0d8869-b6b8-46ea-a4ad-1b327a66c031"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:00:12.239133 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.239100    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-kube-api-access-pfzcb" pod "bf0d8869-b6b8-46ea-a4ad-1b327a66c031" (UID: "bf0d8869-b6b8-46ea-a4ad-1b327a66c031"). InnerVolumeSpecName "kube-api-access-pfzcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:00:12.250777 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.250733    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-metrics-certs" pod "bf0d8869-b6b8-46ea-a4ad-1b327a66c031" (UID: "bf0d8869-b6b8-46ea-a4ad-1b327a66c031"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:00:12.344832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.344796    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:00:12.344832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.344830    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:00:12.344832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.344840    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pfzcb\" (UniqueName: \"kubernetes.io/projected/bf0d8869-b6b8-46ea-a4ad-1b327a66c031-kube-api-access-pfzcb\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:00:12.640907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.640867    6011 generic.go:356] "Generic (PLEG): container finished" podID="bf0d8869-b6b8-46ea-a4ad-1b327a66c031" containerID="01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb" exitCode=0
Oct 30 01:00:12.641117 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.640914    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"
Oct 30 01:00:12.641117 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.640945    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c" event={"ID":"bf0d8869-b6b8-46ea-a4ad-1b327a66c031","Type":"ContainerDied","Data":"01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb"}
Oct 30 01:00:12.641117 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.640995    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c" event={"ID":"bf0d8869-b6b8-46ea-a4ad-1b327a66c031","Type":"ContainerDied","Data":"6bc31f85d4ca2bd364567b056dc718d20a77c8cf2bdf6fb1b73c8e0ceb88ad5c"}
Oct 30 01:00:12.641117 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.641019    6011 scope.go:122] "RemoveContainer" containerID="01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb"
Oct 30 01:00:12.653385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.653365    6011 scope.go:122] "RemoveContainer" containerID="01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb"
Oct 30 01:00:12.653702 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:00:12.653684    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb\": container with ID starting with 01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb not found: ID does not exist" containerID="01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb"
Oct 30 01:00:12.653773 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.653710    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb"} err="failed to get container status \"01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb\": rpc error: code = NotFound desc = could not find container \"01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb\": container with ID starting with 01f0702974a3bd2b4727071dd1716896e0ef754ee3b5b02ceb73873ab4e7d1eb not found: ID does not exist"
Oct 30 01:00:12.700442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.700406    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"]
Oct 30 01:00:12.712804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:12.712765    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-95b31f79-76c9b4fc59-8wv7c"]
Oct 30 01:00:14.057202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:14.052085    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bf0d8869-b6b8-46ea-a4ad-1b327a66c031" path="/var/lib/kubelet/pods/bf0d8869-b6b8-46ea-a4ad-1b327a66c031/volumes"
Oct 30 01:00:27.109240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.109204    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"]
Oct 30 01:00:27.109975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.109956    6011 cpu_manager.go:414] "containerMap: removing container" podUID="bf0d8869-b6b8-46ea-a4ad-1b327a66c031" containerName="maas-api"
Oct 30 01:00:27.109975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.109973    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bf0d8869-b6b8-46ea-a4ad-1b327a66c031" containerName="maas-api"
Oct 30 01:00:27.110153 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.110143    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="bf0d8869-b6b8-46ea-a4ad-1b327a66c031" containerName="maas-api"
Oct 30 01:00:27.115432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.115410    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.129019 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.128981    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"]
Oct 30 01:00:27.136907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.136873    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-869c367e-serving-cert\""
Oct 30 01:00:27.212709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.212669    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829918bc-4243-457f-af64-b99ee37ec732-openshift-service-ca\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.212938 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.212846    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsttf\" (UniqueName: \"kubernetes.io/projected/829918bc-4243-457f-af64-b99ee37ec732-kube-api-access-wsttf\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.213133 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.213104    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-metrics-certs\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.213285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.213265    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.314595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.314555    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-metrics-certs\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.314823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.314635    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.314823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.314747    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829918bc-4243-457f-af64-b99ee37ec732-openshift-service-ca\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.314823 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:00:27.314777    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-serving-cert: secret "maas-api-e2e-mt-869c367e-serving-cert" not found
Oct 30 01:00:27.314994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.314832    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsttf\" (UniqueName: \"kubernetes.io/projected/829918bc-4243-457f-af64-b99ee37ec732-kube-api-access-wsttf\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.314994 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:00:27.314848    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls podName:829918bc-4243-457f-af64-b99ee37ec732 nodeName:}" failed. No retries permitted until 2026-10-30 01:00:27.814831632 +0000 UTC m=+1269.917585581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls") pod "maas-api-e2e-mt-869c367e-79f55cd578-b5pnx" (UID: "829918bc-4243-457f-af64-b99ee37ec732") : secret "maas-api-e2e-mt-869c367e-serving-cert" not found
Oct 30 01:00:27.315556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.315536    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829918bc-4243-457f-af64-b99ee37ec732-openshift-service-ca\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.317274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.317253    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-metrics-certs\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.338056 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.338027    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsttf\" (UniqueName: \"kubernetes.io/projected/829918bc-4243-457f-af64-b99ee37ec732-kube-api-access-wsttf\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.823016 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.822973    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.825644 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:27.825619    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-b5pnx\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:27.912314 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:00:27.912267    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:00:28.034441 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:28.034402    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:28.363741 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:00:28.363711    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod829918bc_4243_457f_af64_b99ee37ec732.slice/crio-9eae390717ba5ea12e179989c45a0b8e377a5f20ea45ab4fc115f882dc1f7a7f WatchSource:0}: Error finding container 9eae390717ba5ea12e179989c45a0b8e377a5f20ea45ab4fc115f882dc1f7a7f: Status 404 returned error can't find the container with id 9eae390717ba5ea12e179989c45a0b8e377a5f20ea45ab4fc115f882dc1f7a7f
Oct 30 01:00:28.364314 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:28.364285    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"]
Oct 30 01:00:28.784540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:28.784507    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx" event={"ID":"829918bc-4243-457f-af64-b99ee37ec732","Type":"ContainerStarted","Data":"9eae390717ba5ea12e179989c45a0b8e377a5f20ea45ab4fc115f882dc1f7a7f"}
Oct 30 01:00:29.797398 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:29.797357    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx" event={"ID":"829918bc-4243-457f-af64-b99ee37ec732","Type":"ContainerStarted","Data":"e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676"}
Oct 30 01:00:29.798293 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:29.798245    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:29.840274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:29.840215    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx" podStartSLOduration=2.456003777 podStartE2EDuration="2.840201126s" totalImagesPullingTime="384.197349ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:29.839434447 +0000 UTC m=+1271.942188421" watchObservedRunningTime="2026-10-30 01:00:29.840201126 +0000 UTC m=+1271.942955161"
Oct 30 01:00:35.812157 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:35.812125    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:00:36.849545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.849508    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"]
Oct 30 01:00:36.854098 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.854074    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:36.872923 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.872889    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"]
Oct 30 01:00:36.921480 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.921430    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-4b973b-serving-cert\""
Oct 30 01:00:36.964879 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.964841    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:36.965096 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.964931    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8064a2-6df2-402b-a87b-f10caa63c39a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:36.965259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.965217    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg2sz\" (UniqueName: \"kubernetes.io/projected/ee8064a2-6df2-402b-a87b-f10caa63c39a-kube-api-access-mg2sz\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:36.965417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:36.965281    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.083237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.080277    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg2sz\" (UniqueName: \"kubernetes.io/projected/ee8064a2-6df2-402b-a87b-f10caa63c39a-kube-api-access-mg2sz\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.083237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.080338    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.083237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.080544    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.083237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.080636    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8064a2-6df2-402b-a87b-f10caa63c39a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.083237 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:00:37.081028    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-serving-cert: secret "maas-api-e2e-shared-b-w3-4b973b-serving-cert" not found
Oct 30 01:00:37.083237 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:00:37.081105    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls podName:ee8064a2-6df2-402b-a87b-f10caa63c39a nodeName:}" failed. No retries permitted until 2026-10-30 01:00:37.581084486 +0000 UTC m=+1279.683838449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls") pod "maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6" (UID: "ee8064a2-6df2-402b-a87b-f10caa63c39a") : secret "maas-api-e2e-shared-b-w3-4b973b-serving-cert" not found
Oct 30 01:00:37.083237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.081430    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8064a2-6df2-402b-a87b-f10caa63c39a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.089732 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.089701    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.102123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.102051    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg2sz\" (UniqueName: \"kubernetes.io/projected/ee8064a2-6df2-402b-a87b-f10caa63c39a-kube-api-access-mg2sz\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.588640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.588583    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.591626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.591595    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:37.771772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:37.771718    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:38.156248 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:38.156109    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"]
Oct 30 01:00:38.919238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:38.918415    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6" event={"ID":"ee8064a2-6df2-402b-a87b-f10caa63c39a","Type":"ContainerStarted","Data":"c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28"}
Oct 30 01:00:38.919238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:38.918526    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6" event={"ID":"ee8064a2-6df2-402b-a87b-f10caa63c39a","Type":"ContainerStarted","Data":"1c50954d601f75c3fad0e9855c49fad6d522ad766575eaccaefd7d42a41578e3"}
Oct 30 01:00:38.919623 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:38.919600    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:38.986201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:38.985808    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6" podStartSLOduration=2.66269943 podStartE2EDuration="2.985784442s" totalImagesPullingTime="323.085012ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:00:38.983578616 +0000 UTC m=+1281.086332587" watchObservedRunningTime="2026-10-30 01:00:38.985784442 +0000 UTC m=+1281.088538416"
Oct 30 01:00:44.933706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:00:44.933675    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:00:57.925437 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:00:57.925400    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:01:09.538294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.538259    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"]
Oct 30 01:01:09.543062 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.543037    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.562377 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.562337    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"]
Oct 30 01:01:09.570710 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.570657    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-fcc0d3e0-serving-cert\""
Oct 30 01:01:09.582642 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.582609    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fljdk\" (UniqueName: \"kubernetes.io/projected/55cd3497-372a-4c8d-9393-544a84dad0dd-kube-api-access-fljdk\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.582795 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.582659    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.582848 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.582801    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-metrics-certs\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.582891 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.582868    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cd3497-372a-4c8d-9393-544a84dad0dd-openshift-service-ca\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.684280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.684243    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fljdk\" (UniqueName: \"kubernetes.io/projected/55cd3497-372a-4c8d-9393-544a84dad0dd-kube-api-access-fljdk\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.684486 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.684289    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.684486 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.684323    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-metrics-certs\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.684486 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.684379    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cd3497-372a-4c8d-9393-544a84dad0dd-openshift-service-ca\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.684486 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:01:09.684452    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-serving-cert: secret "maas-api-e2e-mt-fcc0d3e0-serving-cert" not found
Oct 30 01:01:09.684723 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:01:09.684522    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls podName:55cd3497-372a-4c8d-9393-544a84dad0dd nodeName:}" failed. No retries permitted until 2026-10-30 01:01:10.184504627 +0000 UTC m=+1312.287258575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls") pod "maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz" (UID: "55cd3497-372a-4c8d-9393-544a84dad0dd") : secret "maas-api-e2e-mt-fcc0d3e0-serving-cert" not found
Oct 30 01:01:09.685166 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.685146    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cd3497-372a-4c8d-9393-544a84dad0dd-openshift-service-ca\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.686982 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.686965    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-metrics-certs\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:09.706903 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:09.706875    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fljdk\" (UniqueName: \"kubernetes.io/projected/55cd3497-372a-4c8d-9393-544a84dad0dd-kube-api-access-fljdk\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:10.193705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:10.193674    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:10.196494 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:10.196468    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls\") pod \"maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:10.459503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:10.459413    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:10.763591 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:01:10.763559    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55cd3497_372a_4c8d_9393_544a84dad0dd.slice/crio-70d24eaa34e0fb46a37c7289168511625150386fb0f2f4decb97ffe1b0c001c5 WatchSource:0}: Error finding container 70d24eaa34e0fb46a37c7289168511625150386fb0f2f4decb97ffe1b0c001c5: Status 404 returned error can't find the container with id 70d24eaa34e0fb46a37c7289168511625150386fb0f2f4decb97ffe1b0c001c5
Oct 30 01:01:10.764479 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:10.764428    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"]
Oct 30 01:01:11.503293 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:11.503141    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz" event={"ID":"55cd3497-372a-4c8d-9393-544a84dad0dd","Type":"ContainerStarted","Data":"f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d"}
Oct 30 01:01:11.503861 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:11.503839    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:11.504015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:11.503890    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz" event={"ID":"55cd3497-372a-4c8d-9393-544a84dad0dd","Type":"ContainerStarted","Data":"70d24eaa34e0fb46a37c7289168511625150386fb0f2f4decb97ffe1b0c001c5"}
Oct 30 01:01:11.548742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:11.548684    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz" podStartSLOduration=2.154894385 podStartE2EDuration="2.548669813s" totalImagesPullingTime="393.775428ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:01:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:01:11.545310149 +0000 UTC m=+1313.648064120" watchObservedRunningTime="2026-10-30 01:01:11.548669813 +0000 UTC m=+1313.651423784"
Oct 30 01:01:17.231379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.231336    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"]
Oct 30 01:01:17.236055 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.236032    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.252856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.252659    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"]
Oct 30 01:01:17.262201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.262155    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-7452d0-serving-cert\""
Oct 30 01:01:17.408237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.408204    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-metrics-certs\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.408442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.408337    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-openshift-service-ca\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.408553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.408537    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gjgw\" (UniqueName: \"kubernetes.io/projected/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-kube-api-access-7gjgw\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.408698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.408682    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-maas-api-tls\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.510672 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.510563    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-openshift-service-ca\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.510851 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.510784    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gjgw\" (UniqueName: \"kubernetes.io/projected/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-kube-api-access-7gjgw\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.510958 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.510940    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-maas-api-tls\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.511066 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.511046    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-metrics-certs\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.511587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.511560    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-openshift-service-ca\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.515930 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.515899    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-maas-api-tls\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.520229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.520206    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:01:17.520977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.520951    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-metrics-certs\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.530752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.530730    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gjgw\" (UniqueName: \"kubernetes.io/projected/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-kube-api-access-7gjgw\") pod \"maas-api-e2e-api-a-7452d0-6677558956-qll9l\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.555563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.555538    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:17.885895 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:01:17.885861    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0035acb8_1b9c_419c_95bc_0d9f9da7f0cc.slice/crio-3cc83c20ff559ae73ddc98fdc70f31e2f11338d109331b54900ccbc18f297fd1 WatchSource:0}: Error finding container 3cc83c20ff559ae73ddc98fdc70f31e2f11338d109331b54900ccbc18f297fd1: Status 404 returned error can't find the container with id 3cc83c20ff559ae73ddc98fdc70f31e2f11338d109331b54900ccbc18f297fd1
Oct 30 01:01:17.886562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:17.886535    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"]
Oct 30 01:01:18.558122 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:18.558078    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l" event={"ID":"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc","Type":"ContainerStarted","Data":"e49a7afc7480576cee745f0bc68ad69b5992f121fde9097f08817fc0c803fee6"}
Oct 30 01:01:18.558582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:18.558150    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l" event={"ID":"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc","Type":"ContainerStarted","Data":"3cc83c20ff559ae73ddc98fdc70f31e2f11338d109331b54900ccbc18f297fd1"}
Oct 30 01:01:18.558582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:18.558216    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:18.599042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:18.598990    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l" podStartSLOduration=1.244698022 podStartE2EDuration="1.598977305s" totalImagesPullingTime="354.279283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:01:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:01:18.59850135 +0000 UTC m=+1320.701255321" watchObservedRunningTime="2026-10-30 01:01:18.598977305 +0000 UTC m=+1320.701731276"
Oct 30 01:01:24.572431 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:24.572393    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:01:27.935213 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:01:27.935138    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:01:35.652864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.652830    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"]
Oct 30 01:01:35.657514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.657491    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.673768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.673735    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"]
Oct 30 01:01:35.682379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.682352    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b814fb-serving-cert\""
Oct 30 01:01:35.797477 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.797442    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-metrics-certs\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.797659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.797492    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vsfl\" (UniqueName: \"kubernetes.io/projected/3a13ec78-36b0-4082-b60e-294ac2f21b17-kube-api-access-7vsfl\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.797659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.797616    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-maas-api-tls\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.797746 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.797721    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a13ec78-36b0-4082-b60e-294ac2f21b17-openshift-service-ca\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.899772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.899734    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-metrics-certs\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.899957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.899791    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vsfl\" (UniqueName: \"kubernetes.io/projected/3a13ec78-36b0-4082-b60e-294ac2f21b17-kube-api-access-7vsfl\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.899957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.899843    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-maas-api-tls\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.899957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.899918    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a13ec78-36b0-4082-b60e-294ac2f21b17-openshift-service-ca\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.900605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.900577    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a13ec78-36b0-4082-b60e-294ac2f21b17-openshift-service-ca\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.902451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.902425    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-metrics-certs\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.902571 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.902433    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-maas-api-tls\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.919574 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.919509    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vsfl\" (UniqueName: \"kubernetes.io/projected/3a13ec78-36b0-4082-b60e-294ac2f21b17-kube-api-access-7vsfl\") pod \"maas-api-e2e-api-a-b814fb-7584f4598c-ps54z\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:35.976389 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:35.976351    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:36.303378 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:01:36.303342    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a13ec78_36b0_4082_b60e_294ac2f21b17.slice/crio-1af02648e8b1ab6471cf9c51171d8d6f7ea5cc7424fb7b9d4977897318ade550 WatchSource:0}: Error finding container 1af02648e8b1ab6471cf9c51171d8d6f7ea5cc7424fb7b9d4977897318ade550: Status 404 returned error can't find the container with id 1af02648e8b1ab6471cf9c51171d8d6f7ea5cc7424fb7b9d4977897318ade550
Oct 30 01:01:36.303641 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:36.303621    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"]
Oct 30 01:01:36.709191 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:36.709150    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z" event={"ID":"3a13ec78-36b0-4082-b60e-294ac2f21b17","Type":"ContainerStarted","Data":"1af02648e8b1ab6471cf9c51171d8d6f7ea5cc7424fb7b9d4977897318ade550"}
Oct 30 01:01:37.720133 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:37.720094    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z" event={"ID":"3a13ec78-36b0-4082-b60e-294ac2f21b17","Type":"ContainerStarted","Data":"ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51"}
Oct 30 01:01:37.720600 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:37.720228    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:37.763339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:37.763289    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z" podStartSLOduration=2.404810509 podStartE2EDuration="2.763276112s" totalImagesPullingTime="358.465603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:01:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:01:37.760804337 +0000 UTC m=+1339.863558318" watchObservedRunningTime="2026-10-30 01:01:37.763276112 +0000 UTC m=+1339.866030082"
Oct 30 01:01:43.735293 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:01:43.735256    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:01:57.912334 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:01:57.912297    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:02:10.791898 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:10.791854    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8901-649818"]
Oct 30 01:02:10.798442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:10.798404    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:10.814451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:10.814418    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8901-649818"]
Oct 30 01:02:10.820105 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:10.820080    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-98k27\""
Oct 30 01:02:10.929247 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:10.929208    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8swf\" (UniqueName: \"kubernetes.io/projected/bc149c5b-443b-482f-8cbd-e068ceac76e6-kube-api-access-g8swf\") pod \"test-curl-8901-649818\" (UID: \"bc149c5b-443b-482f-8cbd-e068ceac76e6\") " pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:11.031514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.031476    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8swf\" (UniqueName: \"kubernetes.io/projected/bc149c5b-443b-482f-8cbd-e068ceac76e6-kube-api-access-g8swf\") pod \"test-curl-8901-649818\" (UID: \"bc149c5b-443b-482f-8cbd-e068ceac76e6\") " pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:11.052476 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.052406    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8swf\" (UniqueName: \"kubernetes.io/projected/bc149c5b-443b-482f-8cbd-e068ceac76e6-kube-api-access-g8swf\") pod \"test-curl-8901-649818\" (UID: \"bc149c5b-443b-482f-8cbd-e068ceac76e6\") " pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:11.126412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.126380    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:11.132705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.132680    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"]
Oct 30 01:02:11.137773 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.137748    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.154440 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.154404    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"]
Oct 30 01:02:11.156014 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.155709    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w0-283eb6-serving-cert\""
Oct 30 01:02:11.235747 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.235712    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gprvz\" (UniqueName: \"kubernetes.io/projected/35903ffc-005b-4c93-a491-15fb07a32c2e-kube-api-access-gprvz\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.235931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.235807    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.235931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.235846    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.235931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.235919    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35903ffc-005b-4c93-a491-15fb07a32c2e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.337565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.337474    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gprvz\" (UniqueName: \"kubernetes.io/projected/35903ffc-005b-4c93-a491-15fb07a32c2e-kube-api-access-gprvz\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.337565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.337542    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.337784 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.337574    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.337784 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.337621    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35903ffc-005b-4c93-a491-15fb07a32c2e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.338370 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.338339    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35903ffc-005b-4c93-a491-15fb07a32c2e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.340360 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.340332    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-maas-api-tls\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.340544 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.340521    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-metrics-certs\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.358733 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.358701    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gprvz\" (UniqueName: \"kubernetes.io/projected/35903ffc-005b-4c93-a491-15fb07a32c2e-kube-api-access-gprvz\") pod \"maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.451640 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:02:11.451605    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc149c5b_443b_482f_8cbd_e068ceac76e6.slice/crio-389da62b0227a88c009af94ea47bc74c7313031abe633b18194a1143debf2edd WatchSource:0}: Error finding container 389da62b0227a88c009af94ea47bc74c7313031abe633b18194a1143debf2edd: Status 404 returned error can't find the container with id 389da62b0227a88c009af94ea47bc74c7313031abe633b18194a1143debf2edd
Oct 30 01:02:11.452655 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.452623    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8901-649818"]
Oct 30 01:02:11.491838 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.491816    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:11.800581 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:02:11.800547    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35903ffc_005b_4c93_a491_15fb07a32c2e.slice/crio-e829d2401b3210afc066b9f34490dd89ba97dc4bc5c2b452adf07740802516dc WatchSource:0}: Error finding container e829d2401b3210afc066b9f34490dd89ba97dc4bc5c2b452adf07740802516dc: Status 404 returned error can't find the container with id e829d2401b3210afc066b9f34490dd89ba97dc4bc5c2b452adf07740802516dc
Oct 30 01:02:11.801592 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:11.801569    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"]
Oct 30 01:02:12.031783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:12.031742    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss" event={"ID":"35903ffc-005b-4c93-a491-15fb07a32c2e","Type":"ContainerStarted","Data":"e829d2401b3210afc066b9f34490dd89ba97dc4bc5c2b452adf07740802516dc"}
Oct 30 01:02:12.033388 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:12.033362    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8901-649818" event={"ID":"bc149c5b-443b-482f-8cbd-e068ceac76e6","Type":"ContainerStarted","Data":"afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837"}
Oct 30 01:02:12.033388 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:12.033396    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8901-649818" event={"ID":"bc149c5b-443b-482f-8cbd-e068ceac76e6","Type":"ContainerStarted","Data":"389da62b0227a88c009af94ea47bc74c7313031abe633b18194a1143debf2edd"}
Oct 30 01:02:12.075816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:12.075704    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8901-649818" podStartSLOduration=2.075689122 podStartE2EDuration="2.075689122s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:02:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:02:12.071166325 +0000 UTC m=+1374.173920344" watchObservedRunningTime="2026-10-30 01:02:12.075689122 +0000 UTC m=+1374.178443093"
Oct 30 01:02:12.887277 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:12.887241    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8901-649818"]
Oct 30 01:02:12.899460 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:12.892947    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8901-649818"]
Oct 30 01:02:13.042234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:13.042201    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss" event={"ID":"35903ffc-005b-4c93-a491-15fb07a32c2e","Type":"ContainerStarted","Data":"78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140"}
Oct 30 01:02:13.042426 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:13.042306    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:13.086044 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:13.085992    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss" podStartSLOduration=1.778477771 podStartE2EDuration="2.085978114s" totalImagesPullingTime="307.500343ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:02:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:02:13.082775388 +0000 UTC m=+1375.185529362" watchObservedRunningTime="2026-10-30 01:02:13.085978114 +0000 UTC m=+1375.188732085"
Oct 30 01:02:14.049112 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:14.049053    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8901-649818" podUID="bc149c5b-443b-482f-8cbd-e068ceac76e6" containerName="test-curl-8901-649818" containerID="cri-o://afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837" gracePeriod=2
Oct 30 01:02:16.280933 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:16.280908    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:16.422365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:16.422278    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bc149c5b-443b-482f-8cbd-e068ceac76e6-kube-api-access-g8swf\" (UniqueName: \"kubernetes.io/projected/bc149c5b-443b-482f-8cbd-e068ceac76e6-kube-api-access-g8swf\") pod \"bc149c5b-443b-482f-8cbd-e068ceac76e6\" (UID: \"bc149c5b-443b-482f-8cbd-e068ceac76e6\") "
Oct 30 01:02:16.424701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:16.424673    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc149c5b-443b-482f-8cbd-e068ceac76e6-kube-api-access-g8swf" pod "bc149c5b-443b-482f-8cbd-e068ceac76e6" (UID: "bc149c5b-443b-482f-8cbd-e068ceac76e6"). InnerVolumeSpecName "kube-api-access-g8swf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:02:16.524005 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:16.523967    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8swf\" (UniqueName: \"kubernetes.io/projected/bc149c5b-443b-482f-8cbd-e068ceac76e6-kube-api-access-g8swf\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:02:17.074723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:17.074689    6011 generic.go:356] "Generic (PLEG): container finished" podID="bc149c5b-443b-482f-8cbd-e068ceac76e6" containerID="afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837" exitCode=137
Oct 30 01:02:17.074911 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:17.074735    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:17.074911 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:17.074798    6011 scope.go:122] "RemoveContainer" containerID="afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837"
Oct 30 01:02:17.085787 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:17.085766    6011 scope.go:122] "RemoveContainer" containerID="afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837"
Oct 30 01:02:17.086048 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:02:17.086028    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837\": container with ID starting with afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837 not found: ID does not exist" containerID="afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837"
Oct 30 01:02:17.086126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:17.086056    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837"} err="failed to get container status \"afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837\": rpc error: code = NotFound desc = could not find container \"afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837\": container with ID starting with afee8e54a37504581816cf06fd4921c5286bfe3ea942352bed60eaf3bf997837 not found: ID does not exist"
Oct 30 01:02:17.098388 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:02:17.098363    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8901-649818\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="bc149c5b-443b-482f-8cbd-e068ceac76e6" pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:18.022683 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:18.022650    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bc149c5b-443b-482f-8cbd-e068ceac76e6" path="/var/lib/kubelet/pods/bc149c5b-443b-482f-8cbd-e068ceac76e6/volumes"
Oct 30 01:02:18.033368 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:02:18.033318    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8901-649818\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="bc149c5b-443b-482f-8cbd-e068ceac76e6" pod="opendatahub/test-curl-8901-649818"
Oct 30 01:02:19.054050 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.054022    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:02:19.216689 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.216655    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"]
Oct 30 01:02:19.217399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.217384    6011 cpu_manager.go:414] "containerMap: removing container" podUID="bc149c5b-443b-482f-8cbd-e068ceac76e6" containerName="test-curl-8901-649818"
Oct 30 01:02:19.217446 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.217400    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bc149c5b-443b-482f-8cbd-e068ceac76e6" containerName="test-curl-8901-649818"
Oct 30 01:02:19.217562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.217552    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="bc149c5b-443b-482f-8cbd-e068ceac76e6" containerName="test-curl-8901-649818"
Oct 30 01:02:19.223532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.223501    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.245378 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.245340    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"]
Oct 30 01:02:19.274676 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.274643    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b814fb-openshift-default-dockercfg-mp8rv\""
Oct 30 01:02:19.363669 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.363566    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76bhl\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-kube-api-access-76bhl\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.363842 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.363705    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-socket\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.363842 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.363756    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-credential-socket\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.363961 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.363849    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-certs\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.364016 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.363992    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-envoy\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.364091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.364068    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-token\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.364136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.364110    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-data\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.364312 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.364287    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f756f7be-79d2-4522-83bf-6f733bd9aef9-istiod-ca-cert\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.364495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.364478    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-podinfo\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.466711 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.466660    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-podinfo\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.466909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.466745    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76bhl\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-kube-api-access-76bhl\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.466909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.466802    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-socket\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.466909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.466835    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-credential-socket\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.466909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.466898    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-certs\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.467142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.467041    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-envoy\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.467491 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.467463    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-credential-socket\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.467586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.467514    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-token\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.467586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.467565    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-data\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.467713 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.467597    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-certs\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.467713 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.467638    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f756f7be-79d2-4522-83bf-6f733bd9aef9-istiod-ca-cert\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.467982 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.467873    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-data\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.468159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.468130    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-socket\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.468472 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.468448    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f756f7be-79d2-4522-83bf-6f733bd9aef9-istiod-ca-cert\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.469983 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.469957    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-podinfo\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.470852 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.470828    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-envoy\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.487422 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.487367    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76bhl\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-kube-api-access-76bhl\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.488696 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.488669    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-token\") pod \"e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") " pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.549524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.549483    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:19.933440 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:02:19.933402    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf756f7be_79d2_4522_83bf_6f733bd9aef9.slice/crio-8fb8d7762cde0cf3149a23871c4ca20d7bfd95d50403119b43453daef3b46e2b WatchSource:0}: Error finding container 8fb8d7762cde0cf3149a23871c4ca20d7bfd95d50403119b43453daef3b46e2b: Status 404 returned error can't find the container with id 8fb8d7762cde0cf3149a23871c4ca20d7bfd95d50403119b43453daef3b46e2b
Oct 30 01:02:19.935595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.935568    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"]
Oct 30 01:02:19.937372 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.937329    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:02:19.937502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.937402    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:02:19.937502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:19.937432    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:02:20.110516 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:20.110476    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" event={"ID":"f756f7be-79d2-4522-83bf-6f733bd9aef9","Type":"ContainerStarted","Data":"8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5"}
Oct 30 01:02:20.110928 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:20.110552    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" event={"ID":"f756f7be-79d2-4522-83bf-6f733bd9aef9","Type":"ContainerStarted","Data":"8fb8d7762cde0cf3149a23871c4ca20d7bfd95d50403119b43453daef3b46e2b"}
Oct 30 01:02:20.159672 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:20.159622    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podStartSLOduration=1.159606655 podStartE2EDuration="1.159606655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:02:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:02:20.158636245 +0000 UTC m=+1382.261390216" watchObservedRunningTime="2026-10-30 01:02:20.159606655 +0000 UTC m=+1382.262360626"
Oct 30 01:02:20.555202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:20.553269    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:21.581002 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:21.579380    6011 patch_prober.go:29] interesting pod/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 01:02:21.581002 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:21.579458    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 01:02:22.555109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:22.554636    6011 patch_prober.go:29] interesting pod/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 01:02:22.555109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:22.554721    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 01:02:23.564205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:23.562616    6011 patch_prober.go:29] interesting pod/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 01:02:23.564205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:23.562686    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 01:02:24.218754 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.218719    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"]
Oct 30 01:02:24.223657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.223641    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.261234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.261201    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-30c405-serving-cert\""
Oct 30 01:02:24.275199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.274095    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"]
Oct 30 01:02:24.367349 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.367304    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vr996\" (UniqueName: \"kubernetes.io/projected/928d9066-36f4-4f0e-ac65-00d750a5c09b-kube-api-access-vr996\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.367594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.367564    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928d9066-36f4-4f0e-ac65-00d750a5c09b-openshift-service-ca\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.367662 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.367611    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-metrics-certs\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.367983 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.367960    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-maas-api-tls\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.469259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.469129    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-maas-api-tls\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.469259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.469219    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vr996\" (UniqueName: \"kubernetes.io/projected/928d9066-36f4-4f0e-ac65-00d750a5c09b-kube-api-access-vr996\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.469468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.469358    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928d9066-36f4-4f0e-ac65-00d750a5c09b-openshift-service-ca\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.469468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.469388    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-metrics-certs\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.470163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.470133    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928d9066-36f4-4f0e-ac65-00d750a5c09b-openshift-service-ca\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.471973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.471940    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-maas-api-tls\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.473696 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.473670    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-metrics-certs\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.490223 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.490169    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vr996\" (UniqueName: \"kubernetes.io/projected/928d9066-36f4-4f0e-ac65-00d750a5c09b-kube-api-access-vr996\") pod \"maas-api-e2e-api-b-30c405-5c85c589b4-t27jn\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.550579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.550534    6011 patch_prober.go:29] interesting pod/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 01:02:24.550914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.550875    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 01:02:24.576589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.576551    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:24.908809 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:02:24.908772    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod928d9066_36f4_4f0e_ac65_00d750a5c09b.slice/crio-2ab40964db4dae9f9e2a7652f4e5a535480b5798cd0e77c8e341f36ee0f8199b WatchSource:0}: Error finding container 2ab40964db4dae9f9e2a7652f4e5a535480b5798cd0e77c8e341f36ee0f8199b: Status 404 returned error can't find the container with id 2ab40964db4dae9f9e2a7652f4e5a535480b5798cd0e77c8e341f36ee0f8199b
Oct 30 01:02:24.910645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:24.910619    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"]
Oct 30 01:02:25.185390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:25.185304    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn" event={"ID":"928d9066-36f4-4f0e-ac65-00d750a5c09b","Type":"ContainerStarted","Data":"2ab40964db4dae9f9e2a7652f4e5a535480b5798cd0e77c8e341f36ee0f8199b"}
Oct 30 01:02:25.560848 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:25.560802    6011 patch_prober.go:29] interesting pod/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused" start-of-body=
Oct 30 01:02:25.561145 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:25.561114    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.98:15021/healthz/ready\": dial tcp 10.134.0.98:15021: connect: connection refused"
Oct 30 01:02:26.195340 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:26.195304    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn" event={"ID":"928d9066-36f4-4f0e-ac65-00d750a5c09b","Type":"ContainerStarted","Data":"66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011"}
Oct 30 01:02:26.195740 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:26.195414    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:26.241554 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:26.241481    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn" podStartSLOduration=1.8550889210000001 podStartE2EDuration="2.241458559s" totalImagesPullingTime="386.369638ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:02:26.238289546 +0000 UTC m=+1388.341043542" watchObservedRunningTime="2026-10-30 01:02:26.241458559 +0000 UTC m=+1388.344212533"
Oct 30 01:02:26.561625 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:26.561587    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:26.563619 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:26.563595    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:26.565533 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:26.565494    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:02:27.912646 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:02:27.912597    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:02:32.208910 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:02:32.208876    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:02:57.929372 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:02:57.929311    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:03:19.249771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.249734    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"]
Oct 30 01:03:19.252758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.249969    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx" podUID="829918bc-4243-457f-af64-b99ee37ec732" containerName="maas-api" containerID="cri-o://e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676" gracePeriod=30
Oct 30 01:03:19.586263 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.580619    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:03:19.596476 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.596151    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/829918bc-4243-457f-af64-b99ee37ec732-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829918bc-4243-457f-af64-b99ee37ec732-openshift-service-ca\") pod \"829918bc-4243-457f-af64-b99ee37ec732\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") "
Oct 30 01:03:19.596476 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.596444    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls\") pod \"829918bc-4243-457f-af64-b99ee37ec732\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") "
Oct 30 01:03:19.596748 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.596497    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/829918bc-4243-457f-af64-b99ee37ec732-kube-api-access-wsttf\" (UniqueName: \"kubernetes.io/projected/829918bc-4243-457f-af64-b99ee37ec732-kube-api-access-wsttf\") pod \"829918bc-4243-457f-af64-b99ee37ec732\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") "
Oct 30 01:03:19.596748 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.596547    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-metrics-certs\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-metrics-certs\") pod \"829918bc-4243-457f-af64-b99ee37ec732\" (UID: \"829918bc-4243-457f-af64-b99ee37ec732\") "
Oct 30 01:03:19.596748 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.596632    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/829918bc-4243-457f-af64-b99ee37ec732-openshift-service-ca" pod "829918bc-4243-457f-af64-b99ee37ec732" (UID: "829918bc-4243-457f-af64-b99ee37ec732"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:19.597419 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.597396    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/829918bc-4243-457f-af64-b99ee37ec732-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:19.599944 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.599912    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls" pod "829918bc-4243-457f-af64-b99ee37ec732" (UID: "829918bc-4243-457f-af64-b99ee37ec732"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:19.599944 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.599925    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/829918bc-4243-457f-af64-b99ee37ec732-kube-api-access-wsttf" pod "829918bc-4243-457f-af64-b99ee37ec732" (UID: "829918bc-4243-457f-af64-b99ee37ec732"). InnerVolumeSpecName "kube-api-access-wsttf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:19.600071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.600038    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-metrics-certs" pod "829918bc-4243-457f-af64-b99ee37ec732" (UID: "829918bc-4243-457f-af64-b99ee37ec732"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:19.671051 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.671009    6011 generic.go:356] "Generic (PLEG): container finished" podID="829918bc-4243-457f-af64-b99ee37ec732" containerID="e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676" exitCode=0
Oct 30 01:03:19.671273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.671146    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx" event={"ID":"829918bc-4243-457f-af64-b99ee37ec732","Type":"ContainerDied","Data":"e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676"}
Oct 30 01:03:19.671273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.671219    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx" event={"ID":"829918bc-4243-457f-af64-b99ee37ec732","Type":"ContainerDied","Data":"9eae390717ba5ea12e179989c45a0b8e377a5f20ea45ab4fc115f882dc1f7a7f"}
Oct 30 01:03:19.671273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.671258    6011 scope.go:122] "RemoveContainer" containerID="e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676"
Oct 30 01:03:19.671461 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.671426    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"
Oct 30 01:03:19.691723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.691697    6011 scope.go:122] "RemoveContainer" containerID="e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676"
Oct 30 01:03:19.692086 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:03:19.692059    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676\": container with ID starting with e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676 not found: ID does not exist" containerID="e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676"
Oct 30 01:03:19.692161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.692105    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676"} err="failed to get container status \"e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676\": rpc error: code = NotFound desc = could not find container \"e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676\": container with ID starting with e20564677a4e2d334bb1a766d68578c7313af5a8a279445beecc0aa5bfb45676 not found: ID does not exist"
Oct 30 01:03:19.698596 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.698571    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:19.698596 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.698595    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/829918bc-4243-457f-af64-b99ee37ec732-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:19.698748 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.698604    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsttf\" (UniqueName: \"kubernetes.io/projected/829918bc-4243-457f-af64-b99ee37ec732-kube-api-access-wsttf\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:19.733916 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.733881    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"]
Oct 30 01:03:19.738107 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:19.738076    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-b5pnx"]
Oct 30 01:03:20.023163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:20.023120    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="829918bc-4243-457f-af64-b99ee37ec732" path="/var/lib/kubelet/pods/829918bc-4243-457f-af64-b99ee37ec732/volumes"
Oct 30 01:03:25.265546 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.265491    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"]
Oct 30 01:03:25.267866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.267834    6011 cpu_manager.go:414] "containerMap: removing container" podUID="829918bc-4243-457f-af64-b99ee37ec732" containerName="maas-api"
Oct 30 01:03:25.268103 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.268074    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="829918bc-4243-457f-af64-b99ee37ec732" containerName="maas-api"
Oct 30 01:03:25.268677 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.268643    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="829918bc-4243-457f-af64-b99ee37ec732" containerName="maas-api"
Oct 30 01:03:25.277256 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.277220    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.282471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.282439    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"]
Oct 30 01:03:25.307818 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.307779    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w0-43d152-serving-cert\""
Oct 30 01:03:25.394117 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.394076    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t85hn\" (UniqueName: \"kubernetes.io/projected/405af7db-c0a0-410a-b23f-df70bd1fe1d3-kube-api-access-t85hn\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.394382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.394263    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.394461 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.394406    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/405af7db-c0a0-410a-b23f-df70bd1fe1d3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.394593 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.394576    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.496424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.496387    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t85hn\" (UniqueName: \"kubernetes.io/projected/405af7db-c0a0-410a-b23f-df70bd1fe1d3-kube-api-access-t85hn\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.496640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.496467    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.496640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.496556    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/405af7db-c0a0-410a-b23f-df70bd1fe1d3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.496769 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.496710    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.498149 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.498116    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/405af7db-c0a0-410a-b23f-df70bd1fe1d3-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.505339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.503468    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-metrics-certs\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.506693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.506667    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-maas-api-tls\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.516855 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.516784    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t85hn\" (UniqueName: \"kubernetes.io/projected/405af7db-c0a0-410a-b23f-df70bd1fe1d3-kube-api-access-t85hn\") pod \"maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.596531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.596485    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:25.914062 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:03:25.914020    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod405af7db_c0a0_410a_b23f_df70bd1fe1d3.slice/crio-79fe4dab17824196fe878f115dc52c70a6bdfeff27f7083605df8c7bdb3c00d3 WatchSource:0}: Error finding container 79fe4dab17824196fe878f115dc52c70a6bdfeff27f7083605df8c7bdb3c00d3: Status 404 returned error can't find the container with id 79fe4dab17824196fe878f115dc52c70a6bdfeff27f7083605df8c7bdb3c00d3
Oct 30 01:03:25.916313 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.915851    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:03:25.916313 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:25.916142    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"]
Oct 30 01:03:26.737153 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:26.737052    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j" event={"ID":"405af7db-c0a0-410a-b23f-df70bd1fe1d3","Type":"ContainerStarted","Data":"67d8bcbac621cf9e778fabb20a21666212e262eb70da73d99d358b4b4897ffea"}
Oct 30 01:03:26.737153 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:26.737121    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:26.737153 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:26.737147    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j" event={"ID":"405af7db-c0a0-410a-b23f-df70bd1fe1d3","Type":"ContainerStarted","Data":"79fe4dab17824196fe878f115dc52c70a6bdfeff27f7083605df8c7bdb3c00d3"}
Oct 30 01:03:26.803330 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:26.790474    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j" podStartSLOduration=1.387935569 podStartE2EDuration="1.790450812s" totalImagesPullingTime="402.515243ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:03:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:26.77928626 +0000 UTC m=+1448.882040232" watchObservedRunningTime="2026-10-30 01:03:26.790450812 +0000 UTC m=+1448.893204783"
Oct 30 01:03:27.912884 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:03:27.912799    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:03:31.765660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:31.765619    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"]
Oct 30 01:03:31.766147 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:31.765890    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy" containerID="cri-o://1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef" gracePeriod=30
Oct 30 01:03:32.518703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:32.518666    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"]
Oct 30 01:03:32.518912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:32.518887    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy" containerID="cri-o://c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3" gracePeriod=30
Oct 30 01:03:32.752195 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:32.752149    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:03:37.078543 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.078512    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 01:03:37.200593 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.200550    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-podinfo\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.200845 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.200829    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-data\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.200936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.200924    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-certs\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.200990 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.200963    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-credential-socket\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.201046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.200997    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-token\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-token\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.201097 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201081    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-kube-api-access-jcr4c\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-kube-api-access-jcr4c\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.201159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201116    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-socket\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.201230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201166    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istiod-ca-cert\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.201290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201222    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-data" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.201290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201264    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-certs" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.201400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201307    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-credential-socket" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.201400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201374    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-envoy\") pod \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\" (UID: \"5a1b0938-e3e2-4080-820e-4eddbeb2cb84\") "
Oct 30 01:03:37.201545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.201521    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-socket" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.202325 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.202258    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.202325 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.202285    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.202325 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.202302    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.202325 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.202316    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.202805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.202765    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istiod-ca-cert" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:37.204033 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.204002    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-podinfo" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:03:37.204033 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.204009    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-kube-api-access-jcr4c" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "kube-api-access-jcr4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:37.204244 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.204075    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-token" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:37.205002 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.204973    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-envoy" pod "5a1b0938-e3e2-4080-820e-4eddbeb2cb84" (UID: "5a1b0938-e3e2-4080-820e-4eddbeb2cb84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.304114 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.304017    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.304114 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.304050    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcr4c\" (UniqueName: \"kubernetes.io/projected/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-kube-api-access-jcr4c\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.304114 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.304061    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.304114 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.304081    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.304114 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.304089    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a1b0938-e3e2-4080-820e-4eddbeb2cb84-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.804027 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.804000    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 01:03:37.837975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.837940    6011 generic.go:356] "Generic (PLEG): container finished" podID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerID="c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3" exitCode=0
Oct 30 01:03:37.838156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.837991    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"
Oct 30 01:03:37.838156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.838014    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" event={"ID":"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85","Type":"ContainerDied","Data":"c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3"}
Oct 30 01:03:37.838156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.838099    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww" event={"ID":"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85","Type":"ContainerDied","Data":"712f4443c2384066864716b3b3648d77ef9d029946d51d2c43465dd5e606d23e"}
Oct 30 01:03:37.838156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.838127    6011 scope.go:122] "RemoveContainer" containerID="c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3"
Oct 30 01:03:37.839495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.839474    6011 generic.go:356] "Generic (PLEG): container finished" podID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerID="1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef" exitCode=0
Oct 30 01:03:37.839586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.839530    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"
Oct 30 01:03:37.839586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.839563    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" event={"ID":"5a1b0938-e3e2-4080-820e-4eddbeb2cb84","Type":"ContainerDied","Data":"1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef"}
Oct 30 01:03:37.839662 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.839610    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx" event={"ID":"5a1b0938-e3e2-4080-820e-4eddbeb2cb84","Type":"ContainerDied","Data":"7d0d743d29ebd454c751d2650699ba4478781cd9f7f0d75f7c3377b9f2987d70"}
Oct 30 01:03:37.850419 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.850394    6011 scope.go:122] "RemoveContainer" containerID="c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3"
Oct 30 01:03:37.850758 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:03:37.850738    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3\": container with ID starting with c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3 not found: ID does not exist" containerID="c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3"
Oct 30 01:03:37.850797 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.850769    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3"} err="failed to get container status \"c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3\": rpc error: code = NotFound desc = could not find container \"c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3\": container with ID starting with c3d20c7cccf48f3c8afdbaada14e890c22753769fd1bc779ca4b9ac46d3690c3 not found: ID does not exist"
Oct 30 01:03:37.850797 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.850787    6011 scope.go:122] "RemoveContainer" containerID="1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef"
Oct 30 01:03:37.862877 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.862852    6011 scope.go:122] "RemoveContainer" containerID="1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef"
Oct 30 01:03:37.863329 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:03:37.863298    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef\": container with ID starting with 1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef not found: ID does not exist" containerID="1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef"
Oct 30 01:03:37.863437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.863337    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef"} err="failed to get container status \"1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef\": rpc error: code = NotFound desc = could not find container \"1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef\": container with ID starting with 1735ea338d8f805aff7773028f9cb343046a500a3476fc01c7d22745ca56ceef not found: ID does not exist"
Oct 30 01:03:37.914552 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914456    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-certs\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.914552 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914520    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-data\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.914794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914634    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-token\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-token\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.914794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914673    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-podinfo\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.914794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914710    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-envoy\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.915037 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914795    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-certs" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.915037 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914811    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-data" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.915037 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.914757    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-credential-socket\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.915282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.915131    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-credential-socket" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.915282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.915188    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istiod-ca-cert\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.915569 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.915523    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istiod-ca-cert" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:37.916091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.916068    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-kube-api-access-4gjt5\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-kube-api-access-4gjt5\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.916261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.916162    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-socket\") pod \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\" (UID: \"1004f33b-3dbe-4036-8e4e-6d6bf76a4a85\") "
Oct 30 01:03:37.916814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.916763    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-socket" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.917235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.917168    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.917394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.917242    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.917394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.917256    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.917394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.917270    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.917394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.917282    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:37.918030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.917984    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-podinfo" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:03:37.918314 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.918143    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-envoy" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:37.918407 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.918341    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-token" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:37.919090 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.919065    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-kube-api-access-4gjt5" pod "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" (UID: "1004f33b-3dbe-4036-8e4e-6d6bf76a4a85"). InnerVolumeSpecName "kube-api-access-4gjt5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:37.920635 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.920617    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"]
Oct 30 01:03:37.924278 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:37.924259    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-npbkx"]
Oct 30 01:03:38.019019 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:38.018981    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:38.019019 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:38.019020    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:38.019355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:38.019035    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:38.019355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:38.019051    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gjt5\" (UniqueName: \"kubernetes.io/projected/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85-kube-api-access-4gjt5\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:38.025653 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:38.025626    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" path="/var/lib/kubelet/pods/5a1b0938-e3e2-4080-820e-4eddbeb2cb84/volumes"
Oct 30 01:03:38.182242 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:38.182141    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"]
Oct 30 01:03:38.195157 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:38.195125    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-sqjww"]
Oct 30 01:03:40.022971 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:40.022934    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" path="/var/lib/kubelet/pods/1004f33b-3dbe-4036-8e4e-6d6bf76a4a85/volumes"
Oct 30 01:03:42.797650 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:42.790011    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"]
Oct 30 01:03:42.797650 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:42.790329    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz" podUID="55cd3497-372a-4c8d-9393-544a84dad0dd" containerName="maas-api" containerID="cri-o://f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d" gracePeriod=30
Oct 30 01:03:43.171980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.171925    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:03:43.309375 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.309320    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/55cd3497-372a-4c8d-9393-544a84dad0dd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cd3497-372a-4c8d-9393-544a84dad0dd-openshift-service-ca\") pod \"55cd3497-372a-4c8d-9393-544a84dad0dd\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") "
Oct 30 01:03:43.309563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.309419    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-metrics-certs\") pod \"55cd3497-372a-4c8d-9393-544a84dad0dd\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") "
Oct 30 01:03:43.309563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.309531    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls\") pod \"55cd3497-372a-4c8d-9393-544a84dad0dd\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") "
Oct 30 01:03:43.309719 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.309594    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/55cd3497-372a-4c8d-9393-544a84dad0dd-kube-api-access-fljdk\" (UniqueName: \"kubernetes.io/projected/55cd3497-372a-4c8d-9393-544a84dad0dd-kube-api-access-fljdk\") pod \"55cd3497-372a-4c8d-9393-544a84dad0dd\" (UID: \"55cd3497-372a-4c8d-9393-544a84dad0dd\") "
Oct 30 01:03:43.311723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.311669    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55cd3497-372a-4c8d-9393-544a84dad0dd-openshift-service-ca" pod "55cd3497-372a-4c8d-9393-544a84dad0dd" (UID: "55cd3497-372a-4c8d-9393-544a84dad0dd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:43.314027 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.313992    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-metrics-certs" pod "55cd3497-372a-4c8d-9393-544a84dad0dd" (UID: "55cd3497-372a-4c8d-9393-544a84dad0dd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:43.314196 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.314093    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55cd3497-372a-4c8d-9393-544a84dad0dd-kube-api-access-fljdk" pod "55cd3497-372a-4c8d-9393-544a84dad0dd" (UID: "55cd3497-372a-4c8d-9393-544a84dad0dd"). InnerVolumeSpecName "kube-api-access-fljdk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:43.322601 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.322566    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls" pod "55cd3497-372a-4c8d-9393-544a84dad0dd" (UID: "55cd3497-372a-4c8d-9393-544a84dad0dd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:43.412482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.412385    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:43.412482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.412423    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fljdk\" (UniqueName: \"kubernetes.io/projected/55cd3497-372a-4c8d-9393-544a84dad0dd-kube-api-access-fljdk\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:43.412482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.412437    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/55cd3497-372a-4c8d-9393-544a84dad0dd-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:43.412482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.412449    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/55cd3497-372a-4c8d-9393-544a84dad0dd-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:43.732129 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.732024    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"]
Oct 30 01:03:43.732879 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.732857    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy"
Oct 30 01:03:43.733000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.732880    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy"
Oct 30 01:03:43.733000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.732913    6011 cpu_manager.go:414] "containerMap: removing container" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy"
Oct 30 01:03:43.733000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.732920    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy"
Oct 30 01:03:43.733000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.732943    6011 cpu_manager.go:414] "containerMap: removing container" podUID="55cd3497-372a-4c8d-9393-544a84dad0dd" containerName="maas-api"
Oct 30 01:03:43.733000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.732951    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="55cd3497-372a-4c8d-9393-544a84dad0dd" containerName="maas-api"
Oct 30 01:03:43.733305 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.733121    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a1b0938-e3e2-4080-820e-4eddbeb2cb84" containerName="istio-proxy"
Oct 30 01:03:43.733305 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.733139    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="55cd3497-372a-4c8d-9393-544a84dad0dd" containerName="maas-api"
Oct 30 01:03:43.733305 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.733161    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1004f33b-3dbe-4036-8e4e-6d6bf76a4a85" containerName="istio-proxy"
Oct 30 01:03:43.737309 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.737286    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.754285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.754233    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"]
Oct 30 01:03:43.762341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.762312    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w5-b05110-serving-cert\""
Oct 30 01:03:43.818163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.818120    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.818658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.818335    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd5w6\" (UniqueName: \"kubernetes.io/projected/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-kube-api-access-jd5w6\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.818658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.818520    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.818658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.818627    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.893408 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.893367    6011 generic.go:356] "Generic (PLEG): container finished" podID="55cd3497-372a-4c8d-9393-544a84dad0dd" containerID="f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d" exitCode=0
Oct 30 01:03:43.893579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.893429    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"
Oct 30 01:03:43.893579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.893448    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz" event={"ID":"55cd3497-372a-4c8d-9393-544a84dad0dd","Type":"ContainerDied","Data":"f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d"}
Oct 30 01:03:43.893579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.893504    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz" event={"ID":"55cd3497-372a-4c8d-9393-544a84dad0dd","Type":"ContainerDied","Data":"70d24eaa34e0fb46a37c7289168511625150386fb0f2f4decb97ffe1b0c001c5"}
Oct 30 01:03:43.893579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.893533    6011 scope.go:122] "RemoveContainer" containerID="f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d"
Oct 30 01:03:43.905036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.905015    6011 scope.go:122] "RemoveContainer" containerID="f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d"
Oct 30 01:03:43.905350 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:03:43.905331    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d\": container with ID starting with f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d not found: ID does not exist" containerID="f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d"
Oct 30 01:03:43.905411 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.905359    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d"} err="failed to get container status \"f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d\": rpc error: code = NotFound desc = could not find container \"f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d\": container with ID starting with f02824546a1e513248f002c316c157e3e87fa56d830c87a0753d8e058918ec7d not found: ID does not exist"
Oct 30 01:03:43.919992 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.919968    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.920113 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.920037    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.920232 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.920135    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.920232 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.920217    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd5w6\" (UniqueName: \"kubernetes.io/projected/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-kube-api-access-jd5w6\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.920746 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.920718    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.922603 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.922581    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-metrics-certs\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.922896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.922877    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-maas-api-tls\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.949260 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.949219    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd5w6\" (UniqueName: \"kubernetes.io/projected/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-kube-api-access-jd5w6\") pod \"maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:43.980589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.980548    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"]
Oct 30 01:03:43.991088 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:43.991011    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-fcc0d3e0-84b7ff7865-8jrhz"]
Oct 30 01:03:44.024599 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:44.024555    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="55cd3497-372a-4c8d-9393-544a84dad0dd" path="/var/lib/kubelet/pods/55cd3497-372a-4c8d-9393-544a84dad0dd/volumes"
Oct 30 01:03:44.056682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:44.056645    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:44.383612 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:03:44.383581    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc8d0fc6_ba55_4d6f_bb3b_c31246d1d426.slice/crio-9510e561ec7054720b531a2218b91f6751efa455990d32fd86969e90f9238a9a WatchSource:0}: Error finding container 9510e561ec7054720b531a2218b91f6751efa455990d32fd86969e90f9238a9a: Status 404 returned error can't find the container with id 9510e561ec7054720b531a2218b91f6751efa455990d32fd86969e90f9238a9a
Oct 30 01:03:44.384991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:44.384969    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"]
Oct 30 01:03:44.902488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:44.902454    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg" event={"ID":"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426","Type":"ContainerStarted","Data":"6d1cb9b2e8e2e0a621075964ea541d389536a6ee485cb0609a8a4b3d1c1b195d"}
Oct 30 01:03:44.902938 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:44.902515    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg" event={"ID":"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426","Type":"ContainerStarted","Data":"9510e561ec7054720b531a2218b91f6751efa455990d32fd86969e90f9238a9a"}
Oct 30 01:03:44.902938 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:44.902547    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:44.945332 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:44.945268    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg" podStartSLOduration=1.6197764380000002 podStartE2EDuration="1.945253015s" totalImagesPullingTime="325.476577ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:03:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:44.943830217 +0000 UTC m=+1467.046584188" watchObservedRunningTime="2026-10-30 01:03:44.945253015 +0000 UTC m=+1467.048006985"
Oct 30 01:03:47.023718 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:47.023680    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"]
Oct 30 01:03:47.024238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:47.023914    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy" containerID="cri-o://ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3" gracePeriod=30
Oct 30 01:03:50.923444 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:50.923409    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:03:51.059322 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.059280    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"]
Oct 30 01:03:51.064634 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.064612    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.086690 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.086654    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-869c367e-serving-cert\""
Oct 30 01:03:51.092912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.092876    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"]
Oct 30 01:03:51.131831 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.131789    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frjnc\" (UniqueName: \"kubernetes.io/projected/3f86c08a-744f-45c7-8cd3-b48111f890d0-kube-api-access-frjnc\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.132103 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.132075    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f86c08a-744f-45c7-8cd3-b48111f890d0-openshift-service-ca\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.132278 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.132112    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-metrics-certs\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.132424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.132314    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-maas-api-tls\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.234157 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.234072    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f86c08a-744f-45c7-8cd3-b48111f890d0-openshift-service-ca\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.234157 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.234123    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-metrics-certs\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.234395 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.234298    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-maas-api-tls\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.234511 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.234494    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frjnc\" (UniqueName: \"kubernetes.io/projected/3f86c08a-744f-45c7-8cd3-b48111f890d0-kube-api-access-frjnc\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.234889 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.234866    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f86c08a-744f-45c7-8cd3-b48111f890d0-openshift-service-ca\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.237315 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.237295    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-maas-api-tls\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.237483 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.237465    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-metrics-certs\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.255522 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.255490    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frjnc\" (UniqueName: \"kubernetes.io/projected/3f86c08a-744f-45c7-8cd3-b48111f890d0-kube-api-access-frjnc\") pod \"maas-api-e2e-mt-869c367e-79f55cd578-c9n9m\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.393118 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.393077    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:51.734400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.726104    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"]
Oct 30 01:03:51.987291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:51.986361    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m" event={"ID":"3f86c08a-744f-45c7-8cd3-b48111f890d0","Type":"ContainerStarted","Data":"aaf8fe444980d28196cf064cc0f8f6f5c6e59580389c5d1e88d0b890de196c72"}
Oct 30 01:03:52.393003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.392964    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 01:03:52.487794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.487750    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-kube-api-access-q6pcc\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-kube-api-access-q6pcc\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.487837    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/900090ff-d224-4e19-ad98-9d9d0065ff7b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900090ff-d224-4e19-ad98-9d9d0065ff7b-istiod-ca-cert\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.487965    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-envoy\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.488061    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-certs\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.488091    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-token\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-token\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.488151    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-data\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.488273    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-socket\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.488311    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-credential-socket\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.488637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.488430    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-podinfo\") pod \"900090ff-d224-4e19-ad98-9d9d0065ff7b\" (UID: \"900090ff-d224-4e19-ad98-9d9d0065ff7b\") "
Oct 30 01:03:52.491068 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.490729    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-certs" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.493709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.493647    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/900090ff-d224-4e19-ad98-9d9d0065ff7b-istiod-ca-cert" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:52.494561 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.494530    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-data" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.496991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.496930    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-socket" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.497806 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.497686    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-credential-socket" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.507801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.507714    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-token" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:52.507801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.507728    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-kube-api-access-q6pcc" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "kube-api-access-q6pcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:52.508070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.507810    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-envoy" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:03:52.511815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.511775    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-podinfo" pod "900090ff-d224-4e19-ad98-9d9d0065ff7b" (UID: "900090ff-d224-4e19-ad98-9d9d0065ff7b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:03:52.590557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590514    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.590557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590556    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.590557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590569    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.590557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590582    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.590557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590595    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.590557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590608    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.591108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590622    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q6pcc\" (UniqueName: \"kubernetes.io/projected/900090ff-d224-4e19-ad98-9d9d0065ff7b-kube-api-access-q6pcc\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.591108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590637    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/900090ff-d224-4e19-ad98-9d9d0065ff7b-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.591108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.590651    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/900090ff-d224-4e19-ad98-9d9d0065ff7b-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:52.759756 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.759723    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"]
Oct 30 01:03:52.767269 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:52.760284    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l" podUID="0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" containerName="maas-api" containerID="cri-o://e49a7afc7480576cee745f0bc68ad69b5992f121fde9097f08817fc0c803fee6" gracePeriod=30
Oct 30 01:03:53.017641 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.017559    6011 generic.go:356] "Generic (PLEG): container finished" podID="0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" containerID="e49a7afc7480576cee745f0bc68ad69b5992f121fde9097f08817fc0c803fee6" exitCode=0
Oct 30 01:03:53.018295 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.018198    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l" event={"ID":"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc","Type":"ContainerDied","Data":"e49a7afc7480576cee745f0bc68ad69b5992f121fde9097f08817fc0c803fee6"}
Oct 30 01:03:53.020825 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.020792    6011 generic.go:356] "Generic (PLEG): container finished" podID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerID="ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3" exitCode=0
Oct 30 01:03:53.020933 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.020917    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" event={"ID":"900090ff-d224-4e19-ad98-9d9d0065ff7b","Type":"ContainerDied","Data":"ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3"}
Oct 30 01:03:53.020986 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.020959    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn" event={"ID":"900090ff-d224-4e19-ad98-9d9d0065ff7b","Type":"ContainerDied","Data":"16033dc0f53ce584983102ebd59012a393cbce2f6bc91a93a1790439c78e776d"}
Oct 30 01:03:53.021031 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.020986    6011 scope.go:122] "RemoveContainer" containerID="ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3"
Oct 30 01:03:53.021314 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.021295    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"
Oct 30 01:03:53.031797 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.031763    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m" event={"ID":"3f86c08a-744f-45c7-8cd3-b48111f890d0","Type":"ContainerStarted","Data":"8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f"}
Oct 30 01:03:53.032513 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.032433    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:03:53.065401 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.065370    6011 scope.go:122] "RemoveContainer" containerID="ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3"
Oct 30 01:03:53.066161 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:03:53.066124    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3\": container with ID starting with ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3 not found: ID does not exist" containerID="ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3"
Oct 30 01:03:53.066814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.066231    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3"} err="failed to get container status \"ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3\": rpc error: code = NotFound desc = could not find container \"ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3\": container with ID starting with ac1c457a571efb2852cf7f292d1e815292b2fca8b01b17ee4695e51a91da2dd3 not found: ID does not exist"
Oct 30 01:03:53.105524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.103940    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m" podStartSLOduration=1.73406829 podStartE2EDuration="2.103916541s" totalImagesPullingTime="369.848251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:03:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:03:53.094633119 +0000 UTC m=+1475.197387082" watchObservedRunningTime="2026-10-30 01:03:53.103916541 +0000 UTC m=+1475.206670512"
Oct 30 01:03:53.152490 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.152450    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"]
Oct 30 01:03:53.156717 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.156683    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-s7ldn"]
Oct 30 01:03:53.159712 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.159679    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:03:53.208844 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.208804    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-kube-api-access-7gjgw\" (UniqueName: \"kubernetes.io/projected/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-kube-api-access-7gjgw\") pod \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") "
Oct 30 01:03:53.209048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.208892    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-maas-api-tls\") pod \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") "
Oct 30 01:03:53.209048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.209004    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-openshift-service-ca\") pod \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") "
Oct 30 01:03:53.209163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.209102    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-metrics-certs\") pod \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\" (UID: \"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc\") "
Oct 30 01:03:53.210363 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.210330    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-openshift-service-ca" pod "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" (UID: "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:03:53.213277 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.213241    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-kube-api-access-7gjgw" pod "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" (UID: "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc"). InnerVolumeSpecName "kube-api-access-7gjgw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:03:53.227654 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.226422    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-maas-api-tls" pod "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" (UID: "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:53.227654 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.227601    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-metrics-certs" pod "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" (UID: "0035acb8-1b9c-419c-95bc-0d9f9da7f0cc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:03:53.310820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.310783    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:53.310820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.310812    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:53.310820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.310823    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gjgw\" (UniqueName: \"kubernetes.io/projected/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-kube-api-access-7gjgw\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:53.311066 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:53.310836    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:03:54.023047 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:54.023009    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" path="/var/lib/kubelet/pods/900090ff-d224-4e19-ad98-9d9d0065ff7b/volumes"
Oct 30 01:03:54.044805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:54.044770    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"
Oct 30 01:03:54.044805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:54.044770    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l" event={"ID":"0035acb8-1b9c-419c-95bc-0d9f9da7f0cc","Type":"ContainerDied","Data":"3cc83c20ff559ae73ddc98fdc70f31e2f11338d109331b54900ccbc18f297fd1"}
Oct 30 01:03:54.045050 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:54.044851    6011 scope.go:122] "RemoveContainer" containerID="e49a7afc7480576cee745f0bc68ad69b5992f121fde9097f08817fc0c803fee6"
Oct 30 01:03:54.129656 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:54.129623    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"]
Oct 30 01:03:54.157988 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:54.157955    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-7452d0-6677558956-qll9l"]
Oct 30 01:03:56.023011 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:56.022973    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" path="/var/lib/kubelet/pods/0035acb8-1b9c-419c-95bc-0d9f9da7f0cc/volumes"
Oct 30 01:03:57.911231 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:03:57.911192    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:03:59.051210 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:03:59.051165    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:04:15.379355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.379316    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"]
Oct 30 01:04:15.379815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.379564    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z" podUID="3a13ec78-36b0-4082-b60e-294ac2f21b17" containerName="maas-api" containerID="cri-o://ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51" gracePeriod=30
Oct 30 01:04:15.726275 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.723271    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:04:15.741875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.741732    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-metrics-certs\") pod \"3a13ec78-36b0-4082-b60e-294ac2f21b17\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") "
Oct 30 01:04:15.741875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.741833    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3a13ec78-36b0-4082-b60e-294ac2f21b17-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a13ec78-36b0-4082-b60e-294ac2f21b17-openshift-service-ca\") pod \"3a13ec78-36b0-4082-b60e-294ac2f21b17\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") "
Oct 30 01:04:15.742529 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.742065    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-maas-api-tls\") pod \"3a13ec78-36b0-4082-b60e-294ac2f21b17\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") "
Oct 30 01:04:15.742529 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.742219    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3a13ec78-36b0-4082-b60e-294ac2f21b17-kube-api-access-7vsfl\" (UniqueName: \"kubernetes.io/projected/3a13ec78-36b0-4082-b60e-294ac2f21b17-kube-api-access-7vsfl\") pod \"3a13ec78-36b0-4082-b60e-294ac2f21b17\" (UID: \"3a13ec78-36b0-4082-b60e-294ac2f21b17\") "
Oct 30 01:04:15.742804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.742777    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a13ec78-36b0-4082-b60e-294ac2f21b17-openshift-service-ca" pod "3a13ec78-36b0-4082-b60e-294ac2f21b17" (UID: "3a13ec78-36b0-4082-b60e-294ac2f21b17"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:15.743622 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.742985    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3a13ec78-36b0-4082-b60e-294ac2f21b17-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:15.746241 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.746196    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-maas-api-tls" pod "3a13ec78-36b0-4082-b60e-294ac2f21b17" (UID: "3a13ec78-36b0-4082-b60e-294ac2f21b17"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:15.746241 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.746223    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-metrics-certs" pod "3a13ec78-36b0-4082-b60e-294ac2f21b17" (UID: "3a13ec78-36b0-4082-b60e-294ac2f21b17"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:15.746375 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.746288    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a13ec78-36b0-4082-b60e-294ac2f21b17-kube-api-access-7vsfl" pod "3a13ec78-36b0-4082-b60e-294ac2f21b17" (UID: "3a13ec78-36b0-4082-b60e-294ac2f21b17"). InnerVolumeSpecName "kube-api-access-7vsfl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:15.844629 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.844590    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:15.844629 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.844622    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vsfl\" (UniqueName: \"kubernetes.io/projected/3a13ec78-36b0-4082-b60e-294ac2f21b17-kube-api-access-7vsfl\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:15.844629 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:15.844631    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3a13ec78-36b0-4082-b60e-294ac2f21b17-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:16.283334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.283296    6011 generic.go:356] "Generic (PLEG): container finished" podID="3a13ec78-36b0-4082-b60e-294ac2f21b17" containerID="ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51" exitCode=0
Oct 30 01:04:16.283545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.283348    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"
Oct 30 01:04:16.283545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.283380    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z" event={"ID":"3a13ec78-36b0-4082-b60e-294ac2f21b17","Type":"ContainerDied","Data":"ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51"}
Oct 30 01:04:16.283545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.283477    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z" event={"ID":"3a13ec78-36b0-4082-b60e-294ac2f21b17","Type":"ContainerDied","Data":"1af02648e8b1ab6471cf9c51171d8d6f7ea5cc7424fb7b9d4977897318ade550"}
Oct 30 01:04:16.283545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.283509    6011 scope.go:122] "RemoveContainer" containerID="ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51"
Oct 30 01:04:16.298023 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.297986    6011 scope.go:122] "RemoveContainer" containerID="ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51"
Oct 30 01:04:16.298485 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:04:16.298456    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51\": container with ID starting with ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51 not found: ID does not exist" containerID="ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51"
Oct 30 01:04:16.298603 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.298488    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51"} err="failed to get container status \"ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51\": rpc error: code = NotFound desc = could not find container \"ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51\": container with ID starting with ead248c46ba2c639ef781ecdf62a6abeec26606ef3ee0d40e70683f930b2eb51 not found: ID does not exist"
Oct 30 01:04:16.340465 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.340424    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"]
Oct 30 01:04:16.344011 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:16.343978    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b814fb-7584f4598c-ps54z"]
Oct 30 01:04:18.025265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:18.025232    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3a13ec78-36b0-4082-b60e-294ac2f21b17" path="/var/lib/kubelet/pods/3a13ec78-36b0-4082-b60e-294ac2f21b17/volumes"
Oct 30 01:04:23.703761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:23.703725    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"]
Oct 30 01:04:23.704978 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:23.704022    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy" containerID="cri-o://8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5" gracePeriod=30
Oct 30 01:04:26.565468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:26.565424    6011 patch_prober.go:29] interesting pod/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:04:26.565872 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:26.565482    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:04:27.914143 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:04:27.914093    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:04:28.997894 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:28.997869    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:04:29.171753 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.171705    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-kube-api-access-76bhl\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-kube-api-access-76bhl\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.171965 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.171855    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-credential-socket\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.171965 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.171902    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-envoy\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.171965 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.171939    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-token\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-token\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.172124 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172076    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-data\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.172218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172150    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f756f7be-79d2-4522-83bf-6f733bd9aef9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f756f7be-79d2-4522-83bf-6f733bd9aef9-istiod-ca-cert\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.172218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172199    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-credential-socket" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:29.172352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172244    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-certs\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.172352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172290    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-podinfo\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.172352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172328    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-socket\") pod \"f756f7be-79d2-4522-83bf-6f733bd9aef9\" (UID: \"f756f7be-79d2-4522-83bf-6f733bd9aef9\") "
Oct 30 01:04:29.172617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172587    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-certs" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:29.172709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172614    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f756f7be-79d2-4522-83bf-6f733bd9aef9-istiod-ca-cert" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:29.173022 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.172994    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-data" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:29.173125 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.173097    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-socket" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:29.173429 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.173412    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.173517 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.173431    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.173517 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.173447    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f756f7be-79d2-4522-83bf-6f733bd9aef9-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.173517 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.173458    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.173517 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.173469    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.174542 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.174519    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-token" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:29.174640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.174566    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-podinfo" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:04:29.174769 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.174748    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-envoy" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:04:29.174810 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.174786    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-kube-api-access-76bhl" pod "f756f7be-79d2-4522-83bf-6f733bd9aef9" (UID: "f756f7be-79d2-4522-83bf-6f733bd9aef9"). InnerVolumeSpecName "kube-api-access-76bhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:29.275781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.275699    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.275781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.275741    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.275781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.275756    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f756f7be-79d2-4522-83bf-6f733bd9aef9-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.275781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.275770    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76bhl\" (UniqueName: \"kubernetes.io/projected/f756f7be-79d2-4522-83bf-6f733bd9aef9-kube-api-access-76bhl\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:29.392264 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.392164    6011 generic.go:356] "Generic (PLEG): container finished" podID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerID="8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5" exitCode=0
Oct 30 01:04:29.392264 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.392256    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"
Oct 30 01:04:29.392531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.392293    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" event={"ID":"f756f7be-79d2-4522-83bf-6f733bd9aef9","Type":"ContainerDied","Data":"8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5"}
Oct 30 01:04:29.392531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.392358    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t" event={"ID":"f756f7be-79d2-4522-83bf-6f733bd9aef9","Type":"ContainerDied","Data":"8fb8d7762cde0cf3149a23871c4ca20d7bfd95d50403119b43453daef3b46e2b"}
Oct 30 01:04:29.392531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.392377    6011 scope.go:122] "RemoveContainer" containerID="8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5"
Oct 30 01:04:29.405395 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.405372    6011 scope.go:122] "RemoveContainer" containerID="8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5"
Oct 30 01:04:29.405716 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:04:29.405696    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5\": container with ID starting with 8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5 not found: ID does not exist" containerID="8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5"
Oct 30 01:04:29.405757 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.405728    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5"} err="failed to get container status \"8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5\": rpc error: code = NotFound desc = could not find container \"8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5\": container with ID starting with 8c9ecf0329e2da6be1bc9df838b08ee09904bb18b28899c0b423b5d6f3ceb9b5 not found: ID does not exist"
Oct 30 01:04:29.447815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.447777    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"]
Oct 30 01:04:29.451605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.451573    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b814fb-openshift-default-5775bb6787-c9n9t"]
Oct 30 01:04:29.668130 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.668089    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"]
Oct 30 01:04:29.668951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.668916    6011 cpu_manager.go:414] "containerMap: removing container" podUID="0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" containerName="maas-api"
Oct 30 01:04:29.668951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.668944    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" containerName="maas-api"
Oct 30 01:04:29.668951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.668958    6011 cpu_manager.go:414] "containerMap: removing container" podUID="3a13ec78-36b0-4082-b60e-294ac2f21b17" containerName="maas-api"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.668963    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3a13ec78-36b0-4082-b60e-294ac2f21b17" containerName="maas-api"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.668991    6011 cpu_manager.go:414] "containerMap: removing container" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.668996    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.669015    6011 cpu_manager.go:414] "containerMap: removing container" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.669020    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.669203    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="0035acb8-1b9c-419c-95bc-0d9f9da7f0cc" containerName="maas-api"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.669213    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="3a13ec78-36b0-4082-b60e-294ac2f21b17" containerName="maas-api"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.669228    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" containerName="istio-proxy"
Oct 30 01:04:29.669237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.669241    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="900090ff-d224-4e19-ad98-9d9d0065ff7b" containerName="istio-proxy"
Oct 30 01:04:29.676033 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.675993    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.692892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.692861    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c49c0c-serving-cert\""
Oct 30 01:04:29.694040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.694012    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"]
Oct 30 01:04:29.783540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.783491    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-metrics-certs\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.783786 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.783630    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksgqs\" (UniqueName: \"kubernetes.io/projected/b1f53048-f4c9-4633-9e63-9f3e30348cc2-kube-api-access-ksgqs\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.783786 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.783773    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1f53048-f4c9-4633-9e63-9f3e30348cc2-openshift-service-ca\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.783914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.783898    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.885230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.885154    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.885447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.885318    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-metrics-certs\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.885447 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:04:29.885336    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-serving-cert: secret "maas-api-e2e-api-a-c49c0c-serving-cert" not found
Oct 30 01:04:29.885447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.885379    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksgqs\" (UniqueName: \"kubernetes.io/projected/b1f53048-f4c9-4633-9e63-9f3e30348cc2-kube-api-access-ksgqs\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.885447 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:04:29.885427    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls podName:b1f53048-f4c9-4633-9e63-9f3e30348cc2 nodeName:}" failed. No retries permitted until 2026-10-30 01:04:30.385399273 +0000 UTC m=+1512.488153260 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls") pod "maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj" (UID: "b1f53048-f4c9-4633-9e63-9f3e30348cc2") : secret "maas-api-e2e-api-a-c49c0c-serving-cert" not found
Oct 30 01:04:29.885693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.885577    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1f53048-f4c9-4633-9e63-9f3e30348cc2-openshift-service-ca\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.886493 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.886465    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1f53048-f4c9-4633-9e63-9f3e30348cc2-openshift-service-ca\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.890886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.888757    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-metrics-certs\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:29.909403 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:29.909367    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksgqs\" (UniqueName: \"kubernetes.io/projected/b1f53048-f4c9-4633-9e63-9f3e30348cc2-kube-api-access-ksgqs\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:30.023326 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:30.023227    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f756f7be-79d2-4522-83bf-6f733bd9aef9" path="/var/lib/kubelet/pods/f756f7be-79d2-4522-83bf-6f733bd9aef9/volumes"
Oct 30 01:04:30.393325 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:30.393288    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:30.396191 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:30.396148    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls\") pod \"maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:30.601230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:30.601185    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:30.919935 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:04:30.919891    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f53048_f4c9_4633_9e63_9f3e30348cc2.slice/crio-66ab0f8dde8b7cd73cfa6d486103f9851c2cc07c5775846192604621c8ec6fc9 WatchSource:0}: Error finding container 66ab0f8dde8b7cd73cfa6d486103f9851c2cc07c5775846192604621c8ec6fc9: Status 404 returned error can't find the container with id 66ab0f8dde8b7cd73cfa6d486103f9851c2cc07c5775846192604621c8ec6fc9
Oct 30 01:04:30.920620 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:30.920591    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"]
Oct 30 01:04:31.411404 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:31.411367    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj" event={"ID":"b1f53048-f4c9-4633-9e63-9f3e30348cc2","Type":"ContainerStarted","Data":"481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214"}
Oct 30 01:04:31.411842 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:31.411431    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj" event={"ID":"b1f53048-f4c9-4633-9e63-9f3e30348cc2","Type":"ContainerStarted","Data":"66ab0f8dde8b7cd73cfa6d486103f9851c2cc07c5775846192604621c8ec6fc9"}
Oct 30 01:04:31.411842 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:31.411460    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:31.455456 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:31.455401    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj" podStartSLOduration=2.093081644 podStartE2EDuration="2.455385636s" totalImagesPullingTime="362.303992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:04:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:04:31.451993859 +0000 UTC m=+1513.554747830" watchObservedRunningTime="2026-10-30 01:04:31.455385636 +0000 UTC m=+1513.558139606"
Oct 30 01:04:37.425200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:37.425154    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:04:53.879494 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:53.879458    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"]
Oct 30 01:04:53.879972 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:53.879695    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m" podUID="3f86c08a-744f-45c7-8cd3-b48111f890d0" containerName="maas-api" containerID="cri-o://8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f" gracePeriod=30
Oct 30 01:04:54.045870 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.045829    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m" podUID="3f86c08a-744f-45c7-8cd3-b48111f890d0" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.102:8443/health\": dial tcp 10.134.0.102:8443: connect: connection refused"
Oct 30 01:04:54.219864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.219837    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:04:54.313909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.313868    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-metrics-certs\") pod \"3f86c08a-744f-45c7-8cd3-b48111f890d0\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") "
Oct 30 01:04:54.314149 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.313944    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3f86c08a-744f-45c7-8cd3-b48111f890d0-kube-api-access-frjnc\" (UniqueName: \"kubernetes.io/projected/3f86c08a-744f-45c7-8cd3-b48111f890d0-kube-api-access-frjnc\") pod \"3f86c08a-744f-45c7-8cd3-b48111f890d0\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") "
Oct 30 01:04:54.314149 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.314046    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-maas-api-tls\") pod \"3f86c08a-744f-45c7-8cd3-b48111f890d0\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") "
Oct 30 01:04:54.314306 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.314167    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3f86c08a-744f-45c7-8cd3-b48111f890d0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f86c08a-744f-45c7-8cd3-b48111f890d0-openshift-service-ca\") pod \"3f86c08a-744f-45c7-8cd3-b48111f890d0\" (UID: \"3f86c08a-744f-45c7-8cd3-b48111f890d0\") "
Oct 30 01:04:54.315027 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.314782    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f86c08a-744f-45c7-8cd3-b48111f890d0-openshift-service-ca" pod "3f86c08a-744f-45c7-8cd3-b48111f890d0" (UID: "3f86c08a-744f-45c7-8cd3-b48111f890d0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:04:54.315027 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.314989    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f86c08a-744f-45c7-8cd3-b48111f890d0-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:54.317936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.317901    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-metrics-certs" pod "3f86c08a-744f-45c7-8cd3-b48111f890d0" (UID: "3f86c08a-744f-45c7-8cd3-b48111f890d0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:54.318036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.317916    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-maas-api-tls" pod "3f86c08a-744f-45c7-8cd3-b48111f890d0" (UID: "3f86c08a-744f-45c7-8cd3-b48111f890d0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:04:54.318078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.318050    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f86c08a-744f-45c7-8cd3-b48111f890d0-kube-api-access-frjnc" pod "3f86c08a-744f-45c7-8cd3-b48111f890d0" (UID: "3f86c08a-744f-45c7-8cd3-b48111f890d0"). InnerVolumeSpecName "kube-api-access-frjnc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:04:54.416040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.416005    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:54.416040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.416035    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f86c08a-744f-45c7-8cd3-b48111f890d0-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:54.416270 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.416048    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frjnc\" (UniqueName: \"kubernetes.io/projected/3f86c08a-744f-45c7-8cd3-b48111f890d0-kube-api-access-frjnc\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:04:54.637545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.637507    6011 generic.go:356] "Generic (PLEG): container finished" podID="3f86c08a-744f-45c7-8cd3-b48111f890d0" containerID="8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f" exitCode=0
Oct 30 01:04:54.637841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.637573    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"
Oct 30 01:04:54.637841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.637563    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m" event={"ID":"3f86c08a-744f-45c7-8cd3-b48111f890d0","Type":"ContainerDied","Data":"8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f"}
Oct 30 01:04:54.637841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.637730    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m" event={"ID":"3f86c08a-744f-45c7-8cd3-b48111f890d0","Type":"ContainerDied","Data":"aaf8fe444980d28196cf064cc0f8f6f5c6e59580389c5d1e88d0b890de196c72"}
Oct 30 01:04:54.637841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.637758    6011 scope.go:122] "RemoveContainer" containerID="8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f"
Oct 30 01:04:54.655525 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.655506    6011 scope.go:122] "RemoveContainer" containerID="8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f"
Oct 30 01:04:54.655856 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:04:54.655833    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f\": container with ID starting with 8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f not found: ID does not exist" containerID="8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f"
Oct 30 01:04:54.655906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.655869    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f"} err="failed to get container status \"8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f\": rpc error: code = NotFound desc = could not find container \"8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f\": container with ID starting with 8a1aa152358d227c74097b391e0c899bdf34fc02cb195356cf324d73f446db7f not found: ID does not exist"
Oct 30 01:04:54.695427 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.695394    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"]
Oct 30 01:04:54.703694 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:54.703659    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-869c367e-79f55cd578-c9n9m"]
Oct 30 01:04:56.022317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:04:56.022284    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3f86c08a-744f-45c7-8cd3-b48111f890d0" path="/var/lib/kubelet/pods/3f86c08a-744f-45c7-8cd3-b48111f890d0/volumes"
Oct 30 01:04:57.912481 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:04:57.912440    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:05:05.777303 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:05.777269    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"]
Oct 30 01:05:05.777802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:05.777546    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn" podUID="928d9066-36f4-4f0e-ac65-00d750a5c09b" containerName="maas-api" containerID="cri-o://66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011" gracePeriod=30
Oct 30 01:05:06.095529 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.095497    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:05:06.195731 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.195699    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-metrics-certs\") pod \"928d9066-36f4-4f0e-ac65-00d750a5c09b\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") "
Oct 30 01:05:06.195916 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.195813    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/928d9066-36f4-4f0e-ac65-00d750a5c09b-kube-api-access-vr996\" (UniqueName: \"kubernetes.io/projected/928d9066-36f4-4f0e-ac65-00d750a5c09b-kube-api-access-vr996\") pod \"928d9066-36f4-4f0e-ac65-00d750a5c09b\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") "
Oct 30 01:05:06.195916 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.195890    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/928d9066-36f4-4f0e-ac65-00d750a5c09b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928d9066-36f4-4f0e-ac65-00d750a5c09b-openshift-service-ca\") pod \"928d9066-36f4-4f0e-ac65-00d750a5c09b\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") "
Oct 30 01:05:06.196002 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.195967    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-maas-api-tls\") pod \"928d9066-36f4-4f0e-ac65-00d750a5c09b\" (UID: \"928d9066-36f4-4f0e-ac65-00d750a5c09b\") "
Oct 30 01:05:06.196345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.196308    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/928d9066-36f4-4f0e-ac65-00d750a5c09b-openshift-service-ca" pod "928d9066-36f4-4f0e-ac65-00d750a5c09b" (UID: "928d9066-36f4-4f0e-ac65-00d750a5c09b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:05:06.197041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.196802    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/928d9066-36f4-4f0e-ac65-00d750a5c09b-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:06.198613 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.198586    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-metrics-certs" pod "928d9066-36f4-4f0e-ac65-00d750a5c09b" (UID: "928d9066-36f4-4f0e-ac65-00d750a5c09b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:05:06.198887 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.198865    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-maas-api-tls" pod "928d9066-36f4-4f0e-ac65-00d750a5c09b" (UID: "928d9066-36f4-4f0e-ac65-00d750a5c09b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:05:06.199311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.199281    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/928d9066-36f4-4f0e-ac65-00d750a5c09b-kube-api-access-vr996" pod "928d9066-36f4-4f0e-ac65-00d750a5c09b" (UID: "928d9066-36f4-4f0e-ac65-00d750a5c09b"). InnerVolumeSpecName "kube-api-access-vr996". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:06.298381 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.298341    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vr996\" (UniqueName: \"kubernetes.io/projected/928d9066-36f4-4f0e-ac65-00d750a5c09b-kube-api-access-vr996\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:06.298381 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.298383    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:06.298591 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.298398    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/928d9066-36f4-4f0e-ac65-00d750a5c09b-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:06.744397 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.744365    6011 generic.go:356] "Generic (PLEG): container finished" podID="928d9066-36f4-4f0e-ac65-00d750a5c09b" containerID="66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011" exitCode=0
Oct 30 01:05:06.744587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.744445    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"
Oct 30 01:05:06.744587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.744481    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn" event={"ID":"928d9066-36f4-4f0e-ac65-00d750a5c09b","Type":"ContainerDied","Data":"66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011"}
Oct 30 01:05:06.744587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.744536    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn" event={"ID":"928d9066-36f4-4f0e-ac65-00d750a5c09b","Type":"ContainerDied","Data":"2ab40964db4dae9f9e2a7652f4e5a535480b5798cd0e77c8e341f36ee0f8199b"}
Oct 30 01:05:06.744587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.744575    6011 scope.go:122] "RemoveContainer" containerID="66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011"
Oct 30 01:05:06.756583 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.756555    6011 scope.go:122] "RemoveContainer" containerID="66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011"
Oct 30 01:05:06.756886 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:06.756860    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011\": container with ID starting with 66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011 not found: ID does not exist" containerID="66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011"
Oct 30 01:05:06.756971 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.756899    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011"} err="failed to get container status \"66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011\": rpc error: code = NotFound desc = could not find container \"66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011\": container with ID starting with 66c1b2b4ab3c6130c105cf0f534ca93cdce1c89129e5af91fb0a5e9c143ea011 not found: ID does not exist"
Oct 30 01:05:06.807040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.807005    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"]
Oct 30 01:05:06.813589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:06.813559    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-30c405-5c85c589b4-t27jn"]
Oct 30 01:05:08.022752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:08.022712    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="928d9066-36f4-4f0e-ac65-00d750a5c09b" path="/var/lib/kubelet/pods/928d9066-36f4-4f0e-ac65-00d750a5c09b/volumes"
Oct 30 01:05:27.964431 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:27.964393    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:05:32.960550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.960518    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8898-58b3b1"]
Oct 30 01:05:32.961238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.961223    6011 cpu_manager.go:414] "containerMap: removing container" podUID="3f86c08a-744f-45c7-8cd3-b48111f890d0" containerName="maas-api"
Oct 30 01:05:32.961238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.961239    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3f86c08a-744f-45c7-8cd3-b48111f890d0" containerName="maas-api"
Oct 30 01:05:32.961327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.961262    6011 cpu_manager.go:414] "containerMap: removing container" podUID="928d9066-36f4-4f0e-ac65-00d750a5c09b" containerName="maas-api"
Oct 30 01:05:32.961327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.961267    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="928d9066-36f4-4f0e-ac65-00d750a5c09b" containerName="maas-api"
Oct 30 01:05:32.961392 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.961381    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="3f86c08a-744f-45c7-8cd3-b48111f890d0" containerName="maas-api"
Oct 30 01:05:32.961425 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.961393    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="928d9066-36f4-4f0e-ac65-00d750a5c09b" containerName="maas-api"
Oct 30 01:05:32.966575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.966547    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:32.978127 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.978096    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-58b3b1"]
Oct 30 01:05:32.986384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:32.986342    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-98k27\""
Oct 30 01:05:33.025307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:33.025265    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlvsd\" (UniqueName: \"kubernetes.io/projected/9c3760e7-d62c-4bae-a809-9246091ed4d1-kube-api-access-xlvsd\") pod \"test-curl-8898-58b3b1\" (UID: \"9c3760e7-d62c-4bae-a809-9246091ed4d1\") " pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:33.127072 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:33.127035    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlvsd\" (UniqueName: \"kubernetes.io/projected/9c3760e7-d62c-4bae-a809-9246091ed4d1-kube-api-access-xlvsd\") pod \"test-curl-8898-58b3b1\" (UID: \"9c3760e7-d62c-4bae-a809-9246091ed4d1\") " pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:33.147387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:33.147359    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlvsd\" (UniqueName: \"kubernetes.io/projected/9c3760e7-d62c-4bae-a809-9246091ed4d1-kube-api-access-xlvsd\") pod \"test-curl-8898-58b3b1\" (UID: \"9c3760e7-d62c-4bae-a809-9246091ed4d1\") " pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:33.281850 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:33.281765    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:33.583523 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:05:33.583495    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c3760e7_d62c_4bae_a809_9246091ed4d1.slice/crio-3f86a8e09434f37f57349532cfd3c7aa7c79e4864cbe4286c3d696bd0b8968a9 WatchSource:0}: Error finding container 3f86a8e09434f37f57349532cfd3c7aa7c79e4864cbe4286c3d696bd0b8968a9: Status 404 returned error can't find the container with id 3f86a8e09434f37f57349532cfd3c7aa7c79e4864cbe4286c3d696bd0b8968a9
Oct 30 01:05:33.584308 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:33.584284    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-58b3b1"]
Oct 30 01:05:34.072429 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:34.072390    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-58b3b1" event={"ID":"9c3760e7-d62c-4bae-a809-9246091ed4d1","Type":"ContainerStarted","Data":"b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567"}
Oct 30 01:05:34.072429 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:34.072430    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-58b3b1" event={"ID":"9c3760e7-d62c-4bae-a809-9246091ed4d1","Type":"ContainerStarted","Data":"3f86a8e09434f37f57349532cfd3c7aa7c79e4864cbe4286c3d696bd0b8968a9"}
Oct 30 01:05:34.111624 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:34.111569    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8898-58b3b1" podStartSLOduration=2.111554348 podStartE2EDuration="2.111554348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:34.110222228 +0000 UTC m=+1576.212976198" watchObservedRunningTime="2026-10-30 01:05:34.111554348 +0000 UTC m=+1576.214308319"
Oct 30 01:05:34.957149 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:34.957075    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8898-58b3b1"]
Oct 30 01:05:34.964701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:34.964665    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8898-58b3b1"]
Oct 30 01:05:35.121783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.121753    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8898-310502"]
Oct 30 01:05:35.122470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.122437    6011 cpu_manager.go:414] "containerMap: removing container" podUID="9c3760e7-d62c-4bae-a809-9246091ed4d1" containerName="test-curl-8898-58b3b1"
Oct 30 01:05:35.122470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.122465    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9c3760e7-d62c-4bae-a809-9246091ed4d1" containerName="test-curl-8898-58b3b1"
Oct 30 01:05:35.122716 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.122701    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="9c3760e7-d62c-4bae-a809-9246091ed4d1" containerName="test-curl-8898-58b3b1"
Oct 30 01:05:35.126712 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.126695    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:35.138945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.138917    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-310502"]
Oct 30 01:05:35.256699 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.256615    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4qz2\" (UniqueName: \"kubernetes.io/projected/77e75a84-070f-4993-a7f9-99dd5567f553-kube-api-access-s4qz2\") pod \"test-curl-8898-310502\" (UID: \"77e75a84-070f-4993-a7f9-99dd5567f553\") " pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:35.358429 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.358392    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4qz2\" (UniqueName: \"kubernetes.io/projected/77e75a84-070f-4993-a7f9-99dd5567f553-kube-api-access-s4qz2\") pod \"test-curl-8898-310502\" (UID: \"77e75a84-070f-4993-a7f9-99dd5567f553\") " pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:35.379222 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.379138    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4qz2\" (UniqueName: \"kubernetes.io/projected/77e75a84-070f-4993-a7f9-99dd5567f553-kube-api-access-s4qz2\") pod \"test-curl-8898-310502\" (UID: \"77e75a84-070f-4993-a7f9-99dd5567f553\") " pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:35.446060 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.446026    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:35.748498 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:05:35.748467    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77e75a84_070f_4993_a7f9_99dd5567f553.slice/crio-f3170524cf9b2107d137b0017bb8332ed278acc21e35f147ce16f68e9c82474f WatchSource:0}: Error finding container f3170524cf9b2107d137b0017bb8332ed278acc21e35f147ce16f68e9c82474f: Status 404 returned error can't find the container with id f3170524cf9b2107d137b0017bb8332ed278acc21e35f147ce16f68e9c82474f
Oct 30 01:05:35.749272 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:35.749248    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-310502"]
Oct 30 01:05:36.100090 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:36.099992    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8898-58b3b1" podUID="9c3760e7-d62c-4bae-a809-9246091ed4d1" containerName="test-curl-8898-58b3b1" containerID="cri-o://b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567" gracePeriod=2
Oct 30 01:05:36.100090 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:36.100061    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-310502" event={"ID":"77e75a84-070f-4993-a7f9-99dd5567f553","Type":"ContainerStarted","Data":"2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924"}
Oct 30 01:05:36.100442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:36.100132    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-310502" event={"ID":"77e75a84-070f-4993-a7f9-99dd5567f553","Type":"ContainerStarted","Data":"f3170524cf9b2107d137b0017bb8332ed278acc21e35f147ce16f68e9c82474f"}
Oct 30 01:05:36.141655 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:36.141580    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8898-310502" podStartSLOduration=1.141560809 podStartE2EDuration="1.141560809s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:36.139576154 +0000 UTC m=+1578.242330124" watchObservedRunningTime="2026-10-30 01:05:36.141560809 +0000 UTC m=+1578.244314781"
Oct 30 01:05:37.076956 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.076919    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8898-310502"]
Oct 30 01:05:37.091989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.091953    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8898-310502"]
Oct 30 01:05:37.254515 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.254479    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8898-953ee2"]
Oct 30 01:05:37.255617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.255594    6011 cpu_manager.go:414] "containerMap: removing container" podUID="77e75a84-070f-4993-a7f9-99dd5567f553" containerName="test-curl-8898-310502"
Oct 30 01:05:37.255682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.255622    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="77e75a84-070f-4993-a7f9-99dd5567f553" containerName="test-curl-8898-310502"
Oct 30 01:05:37.255852 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.255840    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="77e75a84-070f-4993-a7f9-99dd5567f553" containerName="test-curl-8898-310502"
Oct 30 01:05:37.260195 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.260157    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:37.283760 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.283731    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-953ee2"]
Oct 30 01:05:37.412660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.412380    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cprvf\" (UniqueName: \"kubernetes.io/projected/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d-kube-api-access-cprvf\") pod \"test-curl-8898-953ee2\" (UID: \"0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d\") " pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:37.514915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.514878    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cprvf\" (UniqueName: \"kubernetes.io/projected/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d-kube-api-access-cprvf\") pod \"test-curl-8898-953ee2\" (UID: \"0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d\") " pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:37.537888 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.537854    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cprvf\" (UniqueName: \"kubernetes.io/projected/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d-kube-api-access-cprvf\") pod \"test-curl-8898-953ee2\" (UID: \"0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d\") " pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:37.579658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.579625    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:37.902254 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:05:37.901967    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a6af2d2_c63f_4e9c_b638_7cfbd4ea818d.slice/crio-82cf0008c1c0693ab287e45f84446e59b3e5dc2afc271d9e018b47eec3eab553 WatchSource:0}: Error finding container 82cf0008c1c0693ab287e45f84446e59b3e5dc2afc271d9e018b47eec3eab553: Status 404 returned error can't find the container with id 82cf0008c1c0693ab287e45f84446e59b3e5dc2afc271d9e018b47eec3eab553
Oct 30 01:05:37.902849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:37.902821    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-953ee2"]
Oct 30 01:05:38.123822 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.123781    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-953ee2" event={"ID":"0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d","Type":"ContainerStarted","Data":"baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f"}
Oct 30 01:05:38.123976 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.123836    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-953ee2" event={"ID":"0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d","Type":"ContainerStarted","Data":"82cf0008c1c0693ab287e45f84446e59b3e5dc2afc271d9e018b47eec3eab553"}
Oct 30 01:05:38.123976 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.123868    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8898-310502" podUID="77e75a84-070f-4993-a7f9-99dd5567f553" containerName="test-curl-8898-310502" containerID="cri-o://2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924" gracePeriod=2
Oct 30 01:05:38.166751 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.166696    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8898-953ee2" podStartSLOduration=1.166681546 podStartE2EDuration="1.166681546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:38.163056029 +0000 UTC m=+1580.265809999" watchObservedRunningTime="2026-10-30 01:05:38.166681546 +0000 UTC m=+1580.269435513"
Oct 30 01:05:38.346023 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.345991    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:38.452779 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.450920    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9c3760e7-d62c-4bae-a809-9246091ed4d1-kube-api-access-xlvsd\" (UniqueName: \"kubernetes.io/projected/9c3760e7-d62c-4bae-a809-9246091ed4d1-kube-api-access-xlvsd\") pod \"9c3760e7-d62c-4bae-a809-9246091ed4d1\" (UID: \"9c3760e7-d62c-4bae-a809-9246091ed4d1\") "
Oct 30 01:05:38.454398 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.454359    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c3760e7-d62c-4bae-a809-9246091ed4d1-kube-api-access-xlvsd" pod "9c3760e7-d62c-4bae-a809-9246091ed4d1" (UID: "9c3760e7-d62c-4bae-a809-9246091ed4d1"). InnerVolumeSpecName "kube-api-access-xlvsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:38.553063 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.553030    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlvsd\" (UniqueName: \"kubernetes.io/projected/9c3760e7-d62c-4bae-a809-9246091ed4d1-kube-api-access-xlvsd\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:38.982411 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.982371    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8898-953ee2"]
Oct 30 01:05:38.985920 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:38.985885    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8898-953ee2"]
Oct 30 01:05:39.125957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.125926    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-8898-2de7aa"]
Oct 30 01:05:39.126718 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.126701    6011 cpu_manager.go:414] "containerMap: removing container" podUID="0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" containerName="test-curl-8898-953ee2"
Oct 30 01:05:39.126793 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.126719    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" containerName="test-curl-8898-953ee2"
Oct 30 01:05:39.126846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.126836    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" containerName="test-curl-8898-953ee2"
Oct 30 01:05:39.130714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.130694    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:39.133243 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.133214    6011 generic.go:356] "Generic (PLEG): container finished" podID="9c3760e7-d62c-4bae-a809-9246091ed4d1" containerID="b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567" exitCode=137
Oct 30 01:05:39.133374 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.133256    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:39.133374 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.133304    6011 scope.go:122] "RemoveContainer" containerID="b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567"
Oct 30 01:05:39.147695 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.147663    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-2de7aa"]
Oct 30 01:05:39.150328 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.150302    6011 scope.go:122] "RemoveContainer" containerID="b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567"
Oct 30 01:05:39.150704 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:39.150680    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567\": container with ID starting with b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567 not found: ID does not exist" containerID="b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567"
Oct 30 01:05:39.150776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.150712    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567"} err="failed to get container status \"b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567\": rpc error: code = NotFound desc = could not find container \"b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567\": container with ID starting with b4d45f2fabfa43031b06ad8d2c215ad1b6a81ce8476ff164249c8aa2e8748567 not found: ID does not exist"
Oct 30 01:05:39.182673 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:39.182628    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8898-58b3b1\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="9c3760e7-d62c-4bae-a809-9246091ed4d1" pod="opendatahub/test-curl-8898-58b3b1"
Oct 30 01:05:39.263517 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.263429    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z75z\" (UniqueName: \"kubernetes.io/projected/d5a50864-4b34-41ec-97b9-65532085f6a9-kube-api-access-7z75z\") pod \"test-curl-8898-2de7aa\" (UID: \"d5a50864-4b34-41ec-97b9-65532085f6a9\") " pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:39.365323 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.365286    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z75z\" (UniqueName: \"kubernetes.io/projected/d5a50864-4b34-41ec-97b9-65532085f6a9-kube-api-access-7z75z\") pod \"test-curl-8898-2de7aa\" (UID: \"d5a50864-4b34-41ec-97b9-65532085f6a9\") " pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:39.385599 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.385572    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z75z\" (UniqueName: \"kubernetes.io/projected/d5a50864-4b34-41ec-97b9-65532085f6a9-kube-api-access-7z75z\") pod \"test-curl-8898-2de7aa\" (UID: \"d5a50864-4b34-41ec-97b9-65532085f6a9\") " pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:39.453483 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.453447    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:39.761370 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:39.760019    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-8898-2de7aa"]
Oct 30 01:05:40.031950 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.031862    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9c3760e7-d62c-4bae-a809-9246091ed4d1" path="/var/lib/kubelet/pods/9c3760e7-d62c-4bae-a809-9246091ed4d1/volumes"
Oct 30 01:05:40.143720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.143687    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-2de7aa" event={"ID":"d5a50864-4b34-41ec-97b9-65532085f6a9","Type":"ContainerStarted","Data":"2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14"}
Oct 30 01:05:40.143812 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.143737    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-8898-2de7aa" event={"ID":"d5a50864-4b34-41ec-97b9-65532085f6a9","Type":"ContainerStarted","Data":"c298418131d28207cede20ddecfb5185e3ce29af405af541dcfc80999b9f5a31"}
Oct 30 01:05:40.143886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.143862    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8898-953ee2" podUID="0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" containerName="test-curl-8898-953ee2" containerID="cri-o://baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f" gracePeriod=2
Oct 30 01:05:40.184651 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.184590    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-8898-2de7aa" podStartSLOduration=1.18457208 podStartE2EDuration="1.18457208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:40.18374769 +0000 UTC m=+1582.286501661" watchObservedRunningTime="2026-10-30 01:05:40.18457208 +0000 UTC m=+1582.287326053"
Oct 30 01:05:40.336202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.336159    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:40.482315 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.482276    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/77e75a84-070f-4993-a7f9-99dd5567f553-kube-api-access-s4qz2\" (UniqueName: \"kubernetes.io/projected/77e75a84-070f-4993-a7f9-99dd5567f553-kube-api-access-s4qz2\") pod \"77e75a84-070f-4993-a7f9-99dd5567f553\" (UID: \"77e75a84-070f-4993-a7f9-99dd5567f553\") "
Oct 30 01:05:40.484963 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.484924    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77e75a84-070f-4993-a7f9-99dd5567f553-kube-api-access-s4qz2" pod "77e75a84-070f-4993-a7f9-99dd5567f553" (UID: "77e75a84-070f-4993-a7f9-99dd5567f553"). InnerVolumeSpecName "kube-api-access-s4qz2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:40.584651 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:40.584557    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4qz2\" (UniqueName: \"kubernetes.io/projected/77e75a84-070f-4993-a7f9-99dd5567f553-kube-api-access-s4qz2\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:41.041000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:41.040962    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-8898-2de7aa"]
Oct 30 01:05:41.047289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:41.047259    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-8898-2de7aa"]
Oct 30 01:05:41.173199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:41.166393    6011 generic.go:356] "Generic (PLEG): container finished" podID="77e75a84-070f-4993-a7f9-99dd5567f553" containerID="2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924" exitCode=137
Oct 30 01:05:41.173199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:41.167409    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:41.173199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:41.167557    6011 scope.go:122] "RemoveContainer" containerID="2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924"
Oct 30 01:05:41.191757 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:41.191737    6011 scope.go:122] "RemoveContainer" containerID="2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924"
Oct 30 01:05:41.192094 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:41.192069    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924\": container with ID starting with 2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924 not found: ID does not exist" containerID="2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924"
Oct 30 01:05:41.192401 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:41.192100    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924"} err="failed to get container status \"2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924\": rpc error: code = NotFound desc = could not find container \"2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924\": container with ID starting with 2bd271f28166f31caedf7e2a3b37c83ea9725cf803c8bea59f4ed445c85b8924 not found: ID does not exist"
Oct 30 01:05:41.198754 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:41.197548    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8898-310502\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="77e75a84-070f-4993-a7f9-99dd5567f553" pod="opendatahub/test-curl-8898-310502"
Oct 30 01:05:42.028276 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:42.028231    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="77e75a84-070f-4993-a7f9-99dd5567f553" path="/var/lib/kubelet/pods/77e75a84-070f-4993-a7f9-99dd5567f553/volumes"
Oct 30 01:05:42.176232 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:42.176195    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-8898-2de7aa" podUID="d5a50864-4b34-41ec-97b9-65532085f6a9" containerName="test-curl-8898-2de7aa" containerID="cri-o://2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14" gracePeriod=2
Oct 30 01:05:42.507940 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:42.507900    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:42.562041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:42.561950    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d-kube-api-access-cprvf\" (UniqueName: \"kubernetes.io/projected/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d-kube-api-access-cprvf\") pod \"0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d\" (UID: \"0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d\") "
Oct 30 01:05:42.564347 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:42.564312    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d-kube-api-access-cprvf" pod "0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" (UID: "0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d"). InnerVolumeSpecName "kube-api-access-cprvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:42.664394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:42.664357    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cprvf\" (UniqueName: \"kubernetes.io/projected/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d-kube-api-access-cprvf\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:43.185155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:43.185117    6011 generic.go:356] "Generic (PLEG): container finished" podID="0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" containerID="baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f" exitCode=137
Oct 30 01:05:43.185155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:43.185159    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:43.185729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:43.185212    6011 scope.go:122] "RemoveContainer" containerID="baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f"
Oct 30 01:05:43.196057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:43.195931    6011 scope.go:122] "RemoveContainer" containerID="baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f"
Oct 30 01:05:43.196252 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:43.196227    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f\": container with ID starting with baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f not found: ID does not exist" containerID="baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f"
Oct 30 01:05:43.196295 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:43.196264    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f"} err="failed to get container status \"baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f\": rpc error: code = NotFound desc = could not find container \"baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f\": container with ID starting with baabaa3ded9e0522e2f8e1eeda852ae10a9ad9c3297059c641a9ba89c5e78a4f not found: ID does not exist"
Oct 30 01:05:43.208319 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:43.208291    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8898-953ee2\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" pod="opendatahub/test-curl-8898-953ee2"
Oct 30 01:05:44.051593 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:44.045873    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d" path="/var/lib/kubelet/pods/0a6af2d2-c63f-4e9c-b638-7cfbd4ea818d/volumes"
Oct 30 01:05:44.415191 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:44.415153    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:44.489862 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:44.489828    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d5a50864-4b34-41ec-97b9-65532085f6a9-kube-api-access-7z75z\" (UniqueName: \"kubernetes.io/projected/d5a50864-4b34-41ec-97b9-65532085f6a9-kube-api-access-7z75z\") pod \"d5a50864-4b34-41ec-97b9-65532085f6a9\" (UID: \"d5a50864-4b34-41ec-97b9-65532085f6a9\") "
Oct 30 01:05:44.492269 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:44.492242    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5a50864-4b34-41ec-97b9-65532085f6a9-kube-api-access-7z75z" pod "d5a50864-4b34-41ec-97b9-65532085f6a9" (UID: "d5a50864-4b34-41ec-97b9-65532085f6a9"). InnerVolumeSpecName "kube-api-access-7z75z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:05:44.592076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:44.592006    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z75z\" (UniqueName: \"kubernetes.io/projected/d5a50864-4b34-41ec-97b9-65532085f6a9-kube-api-access-7z75z\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:05:45.207574 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:45.207524    6011 generic.go:356] "Generic (PLEG): container finished" podID="d5a50864-4b34-41ec-97b9-65532085f6a9" containerID="2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14" exitCode=137
Oct 30 01:05:45.207889 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:45.207830    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:45.208040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:45.207831    6011 scope.go:122] "RemoveContainer" containerID="2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14"
Oct 30 01:05:45.235640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:45.235465    6011 scope.go:122] "RemoveContainer" containerID="2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14"
Oct 30 01:05:45.236001 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:45.235971    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14\": container with ID starting with 2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14 not found: ID does not exist" containerID="2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14"
Oct 30 01:05:45.236084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:45.236015    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14"} err="failed to get container status \"2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14\": rpc error: code = NotFound desc = could not find container \"2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14\": container with ID starting with 2c37cdf9d517426e7a5f5def72c39af08c4f36861f68a9df5a606a58e14d4a14 not found: ID does not exist"
Oct 30 01:05:45.246377 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:45.246336    6011 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-8898-2de7aa\" is forbidden: User \"system:node:ip-10-0-131-254.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-131-254.ec2.internal' and this object" podUID="d5a50864-4b34-41ec-97b9-65532085f6a9" pod="opendatahub/test-curl-8898-2de7aa"
Oct 30 01:05:46.022091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:46.022056    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d5a50864-4b34-41ec-97b9-65532085f6a9" path="/var/lib/kubelet/pods/d5a50864-4b34-41ec-97b9-65532085f6a9/volumes"
Oct 30 01:05:52.843581 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:52.843547    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"]
Oct 30 01:05:52.844268 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:52.844252    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d5a50864-4b34-41ec-97b9-65532085f6a9" containerName="test-curl-8898-2de7aa"
Oct 30 01:05:52.844331 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:52.844268    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d5a50864-4b34-41ec-97b9-65532085f6a9" containerName="test-curl-8898-2de7aa"
Oct 30 01:05:52.844418 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:52.844409    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="d5a50864-4b34-41ec-97b9-65532085f6a9" containerName="test-curl-8898-2de7aa"
Oct 30 01:05:52.849501 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:52.849483    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:52.865637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:52.865605    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"]
Oct 30 01:05:52.911805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:52.911766    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-49d9f1-serving-cert\""
Oct 30 01:05:53.021289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.021251    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-maas-api-tls\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.021505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.021409    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d25b2a-f431-41a4-912b-8b5bdc763a54-openshift-service-ca\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.021505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.021478    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-metrics-certs\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.021625 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.021514    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg56r\" (UniqueName: \"kubernetes.io/projected/89d25b2a-f431-41a4-912b-8b5bdc763a54-kube-api-access-hg56r\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.123080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.122993    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-maas-api-tls\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.123270 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.123146    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d25b2a-f431-41a4-912b-8b5bdc763a54-openshift-service-ca\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.123322 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.123268    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-metrics-certs\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.123322 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.123303    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg56r\" (UniqueName: \"kubernetes.io/projected/89d25b2a-f431-41a4-912b-8b5bdc763a54-kube-api-access-hg56r\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.124131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.124105    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d25b2a-f431-41a4-912b-8b5bdc763a54-openshift-service-ca\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.126317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.126296    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-maas-api-tls\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.126450 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.126429    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-metrics-certs\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.144545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.144520    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg56r\" (UniqueName: \"kubernetes.io/projected/89d25b2a-f431-41a4-912b-8b5bdc763a54-kube-api-access-hg56r\") pod \"maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.166893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.166608    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:53.507943 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:05:53.507909    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d25b2a_f431_41a4_912b_8b5bdc763a54.slice/crio-79a3faa33782bff34c60c50eb200de0b62113e6f480399dd24d842c5bdaba390 WatchSource:0}: Error finding container 79a3faa33782bff34c60c50eb200de0b62113e6f480399dd24d842c5bdaba390: Status 404 returned error can't find the container with id 79a3faa33782bff34c60c50eb200de0b62113e6f480399dd24d842c5bdaba390
Oct 30 01:05:53.508211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:53.508188    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"]
Oct 30 01:05:54.292870 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:54.292823    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78" event={"ID":"89d25b2a-f431-41a4-912b-8b5bdc763a54","Type":"ContainerStarted","Data":"b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb"}
Oct 30 01:05:54.293294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:54.292902    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:05:54.293294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:54.292924    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78" event={"ID":"89d25b2a-f431-41a4-912b-8b5bdc763a54","Type":"ContainerStarted","Data":"79a3faa33782bff34c60c50eb200de0b62113e6f480399dd24d842c5bdaba390"}
Oct 30 01:05:54.336249 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:54.336144    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78" podStartSLOduration=1.952738363 podStartE2EDuration="2.336123668s" totalImagesPullingTime="383.385305ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:05:54.333795559 +0000 UTC m=+1596.436549530" watchObservedRunningTime="2026-10-30 01:05:54.336123668 +0000 UTC m=+1596.438877656"
Oct 30 01:05:57.949501 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:05:57.949461    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:05:58.967134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:58.967097    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"]
Oct 30 01:05:58.971824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:58.971806    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:58.989616 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:58.989587    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"]
Oct 30 01:05:58.999112 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:58.999082    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b483fd-serving-cert\""
Oct 30 01:05:59.033792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.033753    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-metrics-certs\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.033957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.033865    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-maas-api-tls\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.034021 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.033956    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ng4h\" (UniqueName: \"kubernetes.io/projected/ceeb5930-31fd-4f81-9edb-93423e0f621f-kube-api-access-8ng4h\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.034110 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.034090    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceeb5930-31fd-4f81-9edb-93423e0f621f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.138698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.138654    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-metrics-certs\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.138996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.138730    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-maas-api-tls\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.138996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.138828    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ng4h\" (UniqueName: \"kubernetes.io/projected/ceeb5930-31fd-4f81-9edb-93423e0f621f-kube-api-access-8ng4h\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.138996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.138899    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceeb5930-31fd-4f81-9edb-93423e0f621f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.139804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.139775    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceeb5930-31fd-4f81-9edb-93423e0f621f-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.141875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.141808    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-metrics-certs\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.142034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.141937    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-maas-api-tls\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.161280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.161242    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ng4h\" (UniqueName: \"kubernetes.io/projected/ceeb5930-31fd-4f81-9edb-93423e0f621f-kube-api-access-8ng4h\") pod \"maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.288715 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.288687    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:05:59.620768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:05:59.620742    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"]
Oct 30 01:05:59.621400 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:05:59.621370    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceeb5930_31fd_4f81_9edb_93423e0f621f.slice/crio-10752b0be531427a25d9dd3d871f81ad2ba5483ff964d7a0acb92b64b66c3ac5 WatchSource:0}: Error finding container 10752b0be531427a25d9dd3d871f81ad2ba5483ff964d7a0acb92b64b66c3ac5: Status 404 returned error can't find the container with id 10752b0be531427a25d9dd3d871f81ad2ba5483ff964d7a0acb92b64b66c3ac5
Oct 30 01:06:00.311526 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:00.309322    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:06:00.366589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:00.366546    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd" event={"ID":"ceeb5930-31fd-4f81-9edb-93423e0f621f","Type":"ContainerStarted","Data":"c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063"}
Oct 30 01:06:00.366788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:00.366624    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd" event={"ID":"ceeb5930-31fd-4f81-9edb-93423e0f621f","Type":"ContainerStarted","Data":"10752b0be531427a25d9dd3d871f81ad2ba5483ff964d7a0acb92b64b66c3ac5"}
Oct 30 01:06:00.366788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:00.366677    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:06:00.412632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:00.412524    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd" podStartSLOduration=1.917291362 podStartE2EDuration="2.412505112s" totalImagesPullingTime="495.21375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:00.408246918 +0000 UTC m=+1602.511000900" watchObservedRunningTime="2026-10-30 01:06:00.412505112 +0000 UTC m=+1602.515259084"
Oct 30 01:06:06.381080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:06.381050    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:06:16.332604 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.330295    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"]
Oct 30 01:06:16.341260 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.341224    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.353947 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.353919    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"]
Oct 30 01:06:16.363744 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.363685    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-8e8ad1-serving-cert\""
Oct 30 01:06:16.478301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.478269    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-metrics-certs\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.478463 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.478401    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d58fbab-07ee-4078-a279-4494d6479e33-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.478463 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.478453    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ffct\" (UniqueName: \"kubernetes.io/projected/0d58fbab-07ee-4078-a279-4494d6479e33-kube-api-access-4ffct\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.478534 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.478502    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.583720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.582610    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d58fbab-07ee-4078-a279-4494d6479e33-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.583720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.582688    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ffct\" (UniqueName: \"kubernetes.io/projected/0d58fbab-07ee-4078-a279-4494d6479e33-kube-api-access-4ffct\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.583720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.582735    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.583720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.583015    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-metrics-certs\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.585020 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.584988    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d58fbab-07ee-4078-a279-4494d6479e33-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.586737 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.586709    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-metrics-certs\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.587102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.587075    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.609642 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.609612    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ffct\" (UniqueName: \"kubernetes.io/projected/0d58fbab-07ee-4078-a279-4494d6479e33-kube-api-access-4ffct\") pod \"maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.658396 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.658349    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:16.990010 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:06:16.989981    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d58fbab_07ee_4078_a279_4494d6479e33.slice/crio-e7264bf3c3af56622f549d6d663bd10e09540445f61053b73b9d2c179e834b37 WatchSource:0}: Error finding container e7264bf3c3af56622f549d6d663bd10e09540445f61053b73b9d2c179e834b37: Status 404 returned error can't find the container with id e7264bf3c3af56622f549d6d663bd10e09540445f61053b73b9d2c179e834b37
Oct 30 01:06:16.990882 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:16.990858    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"]
Oct 30 01:06:17.526734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:17.526694    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz" event={"ID":"0d58fbab-07ee-4078-a279-4494d6479e33","Type":"ContainerStarted","Data":"9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e"}
Oct 30 01:06:17.527230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:17.526764    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz" event={"ID":"0d58fbab-07ee-4078-a279-4494d6479e33","Type":"ContainerStarted","Data":"e7264bf3c3af56622f549d6d663bd10e09540445f61053b73b9d2c179e834b37"}
Oct 30 01:06:17.527230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:17.526938    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:17.576436 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:17.576386    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz" podStartSLOduration=1.1619702379999999 podStartE2EDuration="1.576371231s" totalImagesPullingTime="414.400993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:17.572004319 +0000 UTC m=+1619.674758292" watchObservedRunningTime="2026-10-30 01:06:17.576371231 +0000 UTC m=+1619.679125248"
Oct 30 01:06:17.801089 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:17.801053    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"]
Oct 30 01:06:17.801333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:17.801312    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy" containerID="cri-o://17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7" gracePeriod=30
Oct 30 01:06:23.084644 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.084269    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 01:06:23.186657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.186622    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-podinfo\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.186840 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.186698    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istiod-ca-cert\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.186840 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.186733    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-envoy\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.186840 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.186807    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-kube-api-access-h46vk\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-kube-api-access-h46vk\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.187278 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187241    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istiod-ca-cert" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:06:23.187390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187303    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-data\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.187462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187427    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-credential-socket\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.187529 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187466    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-socket\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.187636 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187561    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-certs\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.187816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187627    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-data" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:23.187816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187642    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-token\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-token\") pod \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\" (UID: \"66c23c6e-0ea1-4786-bb03-b1c8b4731830\") "
Oct 30 01:06:23.187924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.187811    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-credential-socket" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:23.188048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.188023    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-certs" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:23.188121 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.188039    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-socket" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:23.188532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.188506    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.188632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.188538    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.188632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.188556    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.188632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.188569    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.188632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.188585    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.190313 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.190273    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-envoy" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:06:23.190725 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.190696    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-podinfo" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:06:23.190725 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.190704    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-kube-api-access-h46vk" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "kube-api-access-h46vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:23.191679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.191656    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-token" pod "66c23c6e-0ea1-4786-bb03-b1c8b4731830" (UID: "66c23c6e-0ea1-4786-bb03-b1c8b4731830"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:06:23.290311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.290276    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h46vk\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-kube-api-access-h46vk\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.290311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.290308    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.290311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.290319    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.290563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.290327    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66c23c6e-0ea1-4786-bb03-b1c8b4731830-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:06:23.539896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.539870    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:06:23.576547 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.576513    6011 generic.go:356] "Generic (PLEG): container finished" podID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerID="17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7" exitCode=0
Oct 30 01:06:23.576735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.576557    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"
Oct 30 01:06:23.576735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.576591    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" event={"ID":"66c23c6e-0ea1-4786-bb03-b1c8b4731830","Type":"ContainerDied","Data":"17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7"}
Oct 30 01:06:23.576735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.576649    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb" event={"ID":"66c23c6e-0ea1-4786-bb03-b1c8b4731830","Type":"ContainerDied","Data":"f8c61fac54a8114288a206405b489d87779b8a7f2da66245966083f2b40f1c24"}
Oct 30 01:06:23.576735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.576671    6011 scope.go:122] "RemoveContainer" containerID="17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7"
Oct 30 01:06:23.599360 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.599314    6011 scope.go:122] "RemoveContainer" containerID="17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7"
Oct 30 01:06:23.600817 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:06:23.600785    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7\": container with ID starting with 17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7 not found: ID does not exist" containerID="17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7"
Oct 30 01:06:23.600932 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.600832    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7"} err="failed to get container status \"17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7\": rpc error: code = NotFound desc = could not find container \"17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7\": container with ID starting with 17f6acad43a132abda9b1f8c5114c8bb6cb41011a06a2b69bd4aec5dd2a889e7 not found: ID does not exist"
Oct 30 01:06:23.642074 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.642038    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"]
Oct 30 01:06:23.646617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:23.646585    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-nfjcb"]
Oct 30 01:06:24.024702 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:24.024622    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" path="/var/lib/kubelet/pods/66c23c6e-0ea1-4786-bb03-b1c8b4731830/volumes"
Oct 30 01:06:27.911474 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:06:27.911387    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:06:36.494958 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.494923    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"]
Oct 30 01:06:36.495700 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.495683    6011 cpu_manager.go:414] "containerMap: removing container" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy"
Oct 30 01:06:36.495745 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.495702    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy"
Oct 30 01:06:36.495844 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.495835    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="66c23c6e-0ea1-4786-bb03-b1c8b4731830" containerName="istio-proxy"
Oct 30 01:06:36.501081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.501062    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.514921 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.514898    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"]
Oct 30 01:06:36.524851 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.524820    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-286d36-serving-cert\""
Oct 30 01:06:36.580073 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.580038    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-metrics-certs\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.580290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.580168    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-maas-api-tls\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.580390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.580372    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nftsh\" (UniqueName: \"kubernetes.io/projected/0028edc5-ace8-4e51-aaed-47079626d470-kube-api-access-nftsh\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.580447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.580435    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028edc5-ace8-4e51-aaed-47079626d470-openshift-service-ca\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.682397 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.682358    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028edc5-ace8-4e51-aaed-47079626d470-openshift-service-ca\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.682602 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.682460    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-metrics-certs\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.682602 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.682512    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-maas-api-tls\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.682743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.682616    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nftsh\" (UniqueName: \"kubernetes.io/projected/0028edc5-ace8-4e51-aaed-47079626d470-kube-api-access-nftsh\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.683093 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.683058    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028edc5-ace8-4e51-aaed-47079626d470-openshift-service-ca\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.684997 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.684975    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-metrics-certs\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.685100 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.685065    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-maas-api-tls\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.703948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.703920    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nftsh\" (UniqueName: \"kubernetes.io/projected/0028edc5-ace8-4e51-aaed-47079626d470-kube-api-access-nftsh\") pod \"maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:36.818560 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:36.818523    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:37.144436 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:06:37.144307    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0028edc5_ace8_4e51_aaed_47079626d470.slice/crio-7d1fd1da0dbff2c4f14ebeecdef124df413bc939f56086c0badbd11af7eb7c07 WatchSource:0}: Error finding container 7d1fd1da0dbff2c4f14ebeecdef124df413bc939f56086c0badbd11af7eb7c07: Status 404 returned error can't find the container with id 7d1fd1da0dbff2c4f14ebeecdef124df413bc939f56086c0badbd11af7eb7c07
Oct 30 01:06:37.150674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:37.150644    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"]
Oct 30 01:06:37.687635 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:37.687597    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2" event={"ID":"0028edc5-ace8-4e51-aaed-47079626d470","Type":"ContainerStarted","Data":"ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23"}
Oct 30 01:06:37.688099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:37.687663    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2" event={"ID":"0028edc5-ace8-4e51-aaed-47079626d470","Type":"ContainerStarted","Data":"7d1fd1da0dbff2c4f14ebeecdef124df413bc939f56086c0badbd11af7eb7c07"}
Oct 30 01:06:37.688099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:37.687918    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:37.735732 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:37.735669    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2" podStartSLOduration=1.352134039 podStartE2EDuration="1.73565274s" totalImagesPullingTime="383.518701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:37.732454535 +0000 UTC m=+1639.835208506" watchObservedRunningTime="2026-10-30 01:06:37.73565274 +0000 UTC m=+1639.838406708"
Oct 30 01:06:43.704097 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:43.704065    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:06:51.701223 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.701168    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"]
Oct 30 01:06:51.707430 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.707402    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.722125 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.722095    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"]
Oct 30 01:06:51.729761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.729723    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2ca20f\"/\"openshift-service-ca.crt\""
Oct 30 01:06:51.729995 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.729818    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-2ca20f\"/\"kube-root-ca.crt\""
Oct 30 01:06:51.730071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.729817    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2ca20f\"/\"test-model-2ca20f-kserve-self-signed-certs\""
Oct 30 01:06:51.736750 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.736726    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-2ca20f\"/\"default-dockercfg-sqm2w\""
Oct 30 01:06:51.854443 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.854396    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-model-cache\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.854652 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.854451    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tmp-dir\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.854652 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.854549    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-home\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.854750 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.854736    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-dshm\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.854806 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.854793    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk9ds\" (UniqueName: \"kubernetes.io/projected/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-kube-api-access-jk9ds\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.854910 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.854889    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tls-certs\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.956399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.956296    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-model-cache\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.956399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.956362    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tmp-dir\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.956608 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.956403    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-home\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.956659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.956607    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-dshm\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.956706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.956687    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk9ds\" (UniqueName: \"kubernetes.io/projected/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-kube-api-access-jk9ds\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.956750 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.956741    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tls-certs\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.956805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.956774    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-model-cache\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.957557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.957231    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-home\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.957557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.957445    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tmp-dir\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.959880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.959852    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tls-certs\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.963665 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.963642    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-dshm\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:51.978794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:51.978762    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk9ds\" (UniqueName: \"kubernetes.io/projected/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-kube-api-access-jk9ds\") pod \"test-model-2ca20f-kserve-89cf6d47-5sg97\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") " pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:52.026423 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:52.026385    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:52.346881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:52.346835    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"]
Oct 30 01:06:52.348333 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:06:52.348306    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31a3e6d8_409b_41b9_bd30_40dc7fb9d0f2.slice/crio-68e6ca732da3990b04edfb8d1de920f1d44abe353cb70d56588ce6dd1b0d5dcf WatchSource:0}: Error finding container 68e6ca732da3990b04edfb8d1de920f1d44abe353cb70d56588ce6dd1b0d5dcf: Status 404 returned error can't find the container with id 68e6ca732da3990b04edfb8d1de920f1d44abe353cb70d56588ce6dd1b0d5dcf
Oct 30 01:06:52.806081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:52.806043    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97" event={"ID":"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2","Type":"ContainerStarted","Data":"cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15"}
Oct 30 01:06:52.806081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:52.806086    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97" event={"ID":"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2","Type":"ContainerStarted","Data":"68e6ca732da3990b04edfb8d1de920f1d44abe353cb70d56588ce6dd1b0d5dcf"}
Oct 30 01:06:52.852692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:52.852634    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97" podStartSLOduration=1.852618289 podStartE2EDuration="1.852618289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:06:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:06:52.848709366 +0000 UTC m=+1654.951463333" watchObservedRunningTime="2026-10-30 01:06:52.852618289 +0000 UTC m=+1654.955372308"
Oct 30 01:06:53.027210 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:06:53.027143    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:06:57.911381 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:06:57.911336    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:07:02.027238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:02.027204    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:07:02.040849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:02.040817    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:07:02.836700 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:02.836659    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"]
Oct 30 01:07:02.836952 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:02.836927    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy" containerID="cri-o://960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c" gracePeriod=30
Oct 30 01:07:02.895267 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:02.895239    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:07:08.151558 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.151530    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 01:07:08.243501 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.243467    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/128966aa-cede-48de-a21d-6b0b559ed75d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/128966aa-cede-48de-a21d-6b0b559ed75d-istio-podinfo\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.243687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.243604    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-kube-api-access-k5rzl\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-kube-api-access-k5rzl\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.243738 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.243709    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/128966aa-cede-48de-a21d-6b0b559ed75d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/128966aa-cede-48de-a21d-6b0b559ed75d-istiod-ca-cert\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.243792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.243757    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-istio-token\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-istio-token\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.243853 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.243836    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-envoy\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.243938 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.243909    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-socket\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.244007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.243973    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-certs\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.244079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.244015    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-credential-socket\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.244079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.244058    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-data\") pod \"128966aa-cede-48de-a21d-6b0b559ed75d\" (UID: \"128966aa-cede-48de-a21d-6b0b559ed75d\") "
Oct 30 01:07:08.244457 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.244419    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-socket" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:07:08.244609 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.244584    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-data" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:07:08.244813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.244793    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-certs" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:07:08.245030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.245008    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-credential-socket" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:07:08.245151 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.245051    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/128966aa-cede-48de-a21d-6b0b559ed75d-istiod-ca-cert" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:07:08.245713 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.245688    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/128966aa-cede-48de-a21d-6b0b559ed75d-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.245849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.245832    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.245948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.245934    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.246040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.246026    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.246138 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.246125    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.246626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.246596    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/128966aa-cede-48de-a21d-6b0b559ed75d-istio-podinfo" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:07:08.246801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.246774    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-kube-api-access-k5rzl" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "kube-api-access-k5rzl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:07:08.247321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.247300    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-istio-token" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:07:08.247427 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.247404    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-envoy" pod "128966aa-cede-48de-a21d-6b0b559ed75d" (UID: "128966aa-cede-48de-a21d-6b0b559ed75d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:07:08.347890 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.347857    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/128966aa-cede-48de-a21d-6b0b559ed75d-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.347890 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.347885    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/128966aa-cede-48de-a21d-6b0b559ed75d-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.347890 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.347898    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k5rzl\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-kube-api-access-k5rzl\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.348125 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.347908    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/128966aa-cede-48de-a21d-6b0b559ed75d-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:08.944928 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.944891    6011 generic.go:356] "Generic (PLEG): container finished" podID="128966aa-cede-48de-a21d-6b0b559ed75d" containerID="960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c" exitCode=0
Oct 30 01:07:08.945123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.944952    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"
Oct 30 01:07:08.945123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.944979    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" event={"ID":"128966aa-cede-48de-a21d-6b0b559ed75d","Type":"ContainerDied","Data":"960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c"}
Oct 30 01:07:08.945123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.945044    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx" event={"ID":"128966aa-cede-48de-a21d-6b0b559ed75d","Type":"ContainerDied","Data":"a2aa41e7c7cab3e9246d6a902f583233e0425845b364c13975603ba9008b52ea"}
Oct 30 01:07:08.945123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.945068    6011 scope.go:122] "RemoveContainer" containerID="960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c"
Oct 30 01:07:08.963908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.963592    6011 scope.go:122] "RemoveContainer" containerID="960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c"
Oct 30 01:07:08.978244 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:07:08.974204    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c\": container with ID starting with 960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c not found: ID does not exist" containerID="960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c"
Oct 30 01:07:08.978244 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:08.974279    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c"} err="failed to get container status \"960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c\": rpc error: code = NotFound desc = could not find container \"960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c\": container with ID starting with 960c04b72f101d80d90b919ac2935407e4b3e91c9673bd220240549ebb64fc1c not found: ID does not exist"
Oct 30 01:07:09.021470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:09.021431    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"]
Oct 30 01:07:09.033790 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:09.033752    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-2s4dx"]
Oct 30 01:07:10.022982 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:10.022951    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" path="/var/lib/kubelet/pods/128966aa-cede-48de-a21d-6b0b559ed75d/volumes"
Oct 30 01:07:16.561360 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.561305    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"]
Oct 30 01:07:16.562136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.562117    6011 cpu_manager.go:414] "containerMap: removing container" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy"
Oct 30 01:07:16.562222 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.562138    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy"
Oct 30 01:07:16.562291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.562281    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="128966aa-cede-48de-a21d-6b0b559ed75d" containerName="istio-proxy"
Oct 30 01:07:16.580746 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.578295    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.592290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.592255    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"]
Oct 30 01:07:16.595678 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.595647    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b483fd\"/\"default-dockercfg-qbbd5\""
Oct 30 01:07:16.608354 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.608325    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-b483fd\"/\"test-model-b483fd-kserve-self-signed-certs\""
Oct 30 01:07:16.608354 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.608352    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b483fd\"/\"kube-root-ca.crt\""
Oct 30 01:07:16.608536 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.608326    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-b483fd\"/\"openshift-service-ca.crt\""
Oct 30 01:07:16.682643 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.682607    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd3e120-b2be-4ec1-a33d-0564033820b2-tls-certs\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.682833 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.682671    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m52ds\" (UniqueName: \"kubernetes.io/projected/5fd3e120-b2be-4ec1-a33d-0564033820b2-kube-api-access-m52ds\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.682833 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.682773    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-tmp-dir\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.683034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.682896    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-model-cache\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.683034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.682945    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-home\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.683163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.683148    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-dshm\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.784418 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.784374    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-dshm\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.784674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.784654    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd3e120-b2be-4ec1-a33d-0564033820b2-tls-certs\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.784756 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.784692    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m52ds\" (UniqueName: \"kubernetes.io/projected/5fd3e120-b2be-4ec1-a33d-0564033820b2-kube-api-access-m52ds\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.784756 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.784727    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-tmp-dir\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.784867 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.784768    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-model-cache\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.784942 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.784893    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-home\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.785143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.785118    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-model-cache\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.785423 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.785156    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-tmp-dir\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.785423 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.785216    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-home\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.787000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.786975    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-dshm\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.787387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.787328    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd3e120-b2be-4ec1-a33d-0564033820b2-tls-certs\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.808311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.808276    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m52ds\" (UniqueName: \"kubernetes.io/projected/5fd3e120-b2be-4ec1-a33d-0564033820b2-kube-api-access-m52ds\") pod \"test-model-b483fd-kserve-5bdcf886fc-kqd6f\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") " pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:16.901376 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:16.901290    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:17.219993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:17.219900    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"]
Oct 30 01:07:17.221328 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:07:17.221206    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5fd3e120_b2be_4ec1_a33d_0564033820b2.slice/crio-02925eaad58a07400c8ebd56d9d1db4a3709591a52f90257fe7d8e7bc1074ed6 WatchSource:0}: Error finding container 02925eaad58a07400c8ebd56d9d1db4a3709591a52f90257fe7d8e7bc1074ed6: Status 404 returned error can't find the container with id 02925eaad58a07400c8ebd56d9d1db4a3709591a52f90257fe7d8e7bc1074ed6
Oct 30 01:07:18.030474 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:18.030442    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f" event={"ID":"5fd3e120-b2be-4ec1-a33d-0564033820b2","Type":"ContainerStarted","Data":"afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034"}
Oct 30 01:07:18.030474 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:18.030481    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f" event={"ID":"5fd3e120-b2be-4ec1-a33d-0564033820b2","Type":"ContainerStarted","Data":"02925eaad58a07400c8ebd56d9d1db4a3709591a52f90257fe7d8e7bc1074ed6"}
Oct 30 01:07:18.073614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:18.073558    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f" podStartSLOduration=2.073544164 podStartE2EDuration="2.073544164s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:07:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:07:18.071019485 +0000 UTC m=+1680.173773473" watchObservedRunningTime="2026-10-30 01:07:18.073544164 +0000 UTC m=+1680.176298163"
Oct 30 01:07:18.901446 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:18.901351    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:26.902190 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:26.902142    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:26.914955 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:26.914924    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:27.162010 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:27.161905    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:07:27.911848 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:07:27.911805    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:07:32.219432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.219396    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"]
Oct 30 01:07:32.225551 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.225525    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.242064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.242036    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-1eae0b-serving-cert\""
Oct 30 01:07:32.260700 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.260667    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"]
Oct 30 01:07:32.350757 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.350711    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.350977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.350812    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-metrics-certs\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.351055 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.351031    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfmrm\" (UniqueName: \"kubernetes.io/projected/065bcf9d-e7a8-46a7-b730-366f7856f4e4-kube-api-access-gfmrm\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.351300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.351274    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/065bcf9d-e7a8-46a7-b730-366f7856f4e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.454514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.452796    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/065bcf9d-e7a8-46a7-b730-366f7856f4e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.454514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.452871    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.454514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.452913    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-metrics-certs\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.454514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.453057    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfmrm\" (UniqueName: \"kubernetes.io/projected/065bcf9d-e7a8-46a7-b730-366f7856f4e4-kube-api-access-gfmrm\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.454514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.454151    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/065bcf9d-e7a8-46a7-b730-366f7856f4e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.456793 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.456765    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.456914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.456888    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-metrics-certs\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.474364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.474279    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfmrm\" (UniqueName: \"kubernetes.io/projected/065bcf9d-e7a8-46a7-b730-366f7856f4e4-kube-api-access-gfmrm\") pod \"maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.541416 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.541370    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:32.850378 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:07:32.850350    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065bcf9d_e7a8_46a7_b730_366f7856f4e4.slice/crio-bac5eb6dfbb63e31ad35fd809d12619629287ebd5c4bf6e583bb0cea212aa28f WatchSource:0}: Error finding container bac5eb6dfbb63e31ad35fd809d12619629287ebd5c4bf6e583bb0cea212aa28f: Status 404 returned error can't find the container with id bac5eb6dfbb63e31ad35fd809d12619629287ebd5c4bf6e583bb0cea212aa28f
Oct 30 01:07:32.851133 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:32.851108    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"]
Oct 30 01:07:33.196924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:33.196845    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm" event={"ID":"065bcf9d-e7a8-46a7-b730-366f7856f4e4","Type":"ContainerStarted","Data":"bac5eb6dfbb63e31ad35fd809d12619629287ebd5c4bf6e583bb0cea212aa28f"}
Oct 30 01:07:34.206184 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:34.206146    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm" event={"ID":"065bcf9d-e7a8-46a7-b730-366f7856f4e4","Type":"ContainerStarted","Data":"3c470600e5b6b10fa629ec58858c162d72693f377c3c925cfdd8cd72a8faada4"}
Oct 30 01:07:34.206628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:34.206249    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:34.248801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:34.248747    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm" podStartSLOduration=1.9002335339999998 podStartE2EDuration="2.248732246s" totalImagesPullingTime="348.498712ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:07:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:07:34.247447657 +0000 UTC m=+1696.350201628" watchObservedRunningTime="2026-10-30 01:07:34.248732246 +0000 UTC m=+1696.351486216"
Oct 30 01:07:40.220329 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:40.220300    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:07:47.491311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.491169    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"]
Oct 30 01:07:47.491812 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.491563    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj" podUID="b1f53048-f4c9-4633-9e63-9f3e30348cc2" containerName="maas-api" containerID="cri-o://481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214" gracePeriod=30
Oct 30 01:07:47.829286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.829217    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:07:47.926361 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.926311    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1f53048-f4c9-4633-9e63-9f3e30348cc2-kube-api-access-ksgqs\" (UniqueName: \"kubernetes.io/projected/b1f53048-f4c9-4633-9e63-9f3e30348cc2-kube-api-access-ksgqs\") pod \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") "
Oct 30 01:07:47.926745 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.926528    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls\") pod \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") "
Oct 30 01:07:47.927058 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.927034    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-metrics-certs\") pod \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") "
Oct 30 01:07:47.927216 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.927147    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1f53048-f4c9-4633-9e63-9f3e30348cc2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1f53048-f4c9-4633-9e63-9f3e30348cc2-openshift-service-ca\") pod \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\" (UID: \"b1f53048-f4c9-4633-9e63-9f3e30348cc2\") "
Oct 30 01:07:47.927644 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.927612    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1f53048-f4c9-4633-9e63-9f3e30348cc2-openshift-service-ca" pod "b1f53048-f4c9-4633-9e63-9f3e30348cc2" (UID: "b1f53048-f4c9-4633-9e63-9f3e30348cc2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:07:47.928468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.928447    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1f53048-f4c9-4633-9e63-9f3e30348cc2-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:47.931100 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.930982    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls" pod "b1f53048-f4c9-4633-9e63-9f3e30348cc2" (UID: "b1f53048-f4c9-4633-9e63-9f3e30348cc2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:07:47.932531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.932508    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f53048-f4c9-4633-9e63-9f3e30348cc2-kube-api-access-ksgqs" pod "b1f53048-f4c9-4633-9e63-9f3e30348cc2" (UID: "b1f53048-f4c9-4633-9e63-9f3e30348cc2"). InnerVolumeSpecName "kube-api-access-ksgqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:07:47.932621 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:47.932585    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-metrics-certs" pod "b1f53048-f4c9-4633-9e63-9f3e30348cc2" (UID: "b1f53048-f4c9-4633-9e63-9f3e30348cc2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:07:48.029896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.029853    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:48.029896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.029888    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1f53048-f4c9-4633-9e63-9f3e30348cc2-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:48.029896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.029901    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ksgqs\" (UniqueName: \"kubernetes.io/projected/b1f53048-f4c9-4633-9e63-9f3e30348cc2-kube-api-access-ksgqs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:07:48.327004 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.326963    6011 generic.go:356] "Generic (PLEG): container finished" podID="b1f53048-f4c9-4633-9e63-9f3e30348cc2" containerID="481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214" exitCode=0
Oct 30 01:07:48.327004 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.327007    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj" event={"ID":"b1f53048-f4c9-4633-9e63-9f3e30348cc2","Type":"ContainerDied","Data":"481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214"}
Oct 30 01:07:48.327301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.327013    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"
Oct 30 01:07:48.327301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.327060    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj" event={"ID":"b1f53048-f4c9-4633-9e63-9f3e30348cc2","Type":"ContainerDied","Data":"66ab0f8dde8b7cd73cfa6d486103f9851c2cc07c5775846192604621c8ec6fc9"}
Oct 30 01:07:48.327301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.327102    6011 scope.go:122] "RemoveContainer" containerID="481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214"
Oct 30 01:07:48.338926 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.338901    6011 scope.go:122] "RemoveContainer" containerID="481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214"
Oct 30 01:07:48.339292 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:07:48.339266    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214\": container with ID starting with 481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214 not found: ID does not exist" containerID="481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214"
Oct 30 01:07:48.339374 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.339299    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214"} err="failed to get container status \"481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214\": rpc error: code = NotFound desc = could not find container \"481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214\": container with ID starting with 481e32adf97f9cebba72a115d2e4d8f5ab0c10b78902a2db9fad11f39064c214 not found: ID does not exist"
Oct 30 01:07:48.382504 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.382455    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"]
Oct 30 01:07:48.397113 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:48.397057    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c49c0c-86d685d886-5l8kj"]
Oct 30 01:07:50.023836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:50.023802    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1f53048-f4c9-4633-9e63-9f3e30348cc2" path="/var/lib/kubelet/pods/b1f53048-f4c9-4633-9e63-9f3e30348cc2/volumes"
Oct 30 01:07:57.932951 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:07:57.932900    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:07:59.584316 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.584283    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"]
Oct 30 01:07:59.585042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.585027    6011 cpu_manager.go:414] "containerMap: removing container" podUID="b1f53048-f4c9-4633-9e63-9f3e30348cc2" containerName="maas-api"
Oct 30 01:07:59.585097 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.585044    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1f53048-f4c9-4633-9e63-9f3e30348cc2" containerName="maas-api"
Oct 30 01:07:59.585204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.585191    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1f53048-f4c9-4633-9e63-9f3e30348cc2" containerName="maas-api"
Oct 30 01:07:59.590388 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.590369    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.607832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.607803    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"]
Oct 30 01:07:59.614518 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.614333    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-b2bf8a-serving-cert\""
Oct 30 01:07:59.636045 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.636005    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/601a1248-dcd8-4100-bead-8888cb2b1622-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.636286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.636192    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-metrics-certs\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.636382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.636293    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.636382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.636367    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdj6d\" (UniqueName: \"kubernetes.io/projected/601a1248-dcd8-4100-bead-8888cb2b1622-kube-api-access-tdj6d\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.737726 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.737683    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-metrics-certs\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.737931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.737745    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.737931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.737791    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdj6d\" (UniqueName: \"kubernetes.io/projected/601a1248-dcd8-4100-bead-8888cb2b1622-kube-api-access-tdj6d\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.737931 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:07:59.737841    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-serving-cert: secret "maas-api-e2e-inf-b-b2bf8a-serving-cert" not found
Oct 30 01:07:59.737931 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:07:59.737908    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls podName:601a1248-dcd8-4100-bead-8888cb2b1622 nodeName:}" failed. No retries permitted until 2026-10-30 01:08:00.237890517 +0000 UTC m=+1722.340644465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls") pod "maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx" (UID: "601a1248-dcd8-4100-bead-8888cb2b1622") : secret "maas-api-e2e-inf-b-b2bf8a-serving-cert" not found
Oct 30 01:07:59.738094 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.738014    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/601a1248-dcd8-4100-bead-8888cb2b1622-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.738656 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.738620    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/601a1248-dcd8-4100-bead-8888cb2b1622-openshift-service-ca\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.740452 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.740432    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-metrics-certs\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:07:59.761513 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:07:59.761474    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdj6d\" (UniqueName: \"kubernetes.io/projected/601a1248-dcd8-4100-bead-8888cb2b1622-kube-api-access-tdj6d\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:08:00.288506 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:00.288456    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:08:00.291626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:00.291592    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls\") pod \"maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:08:00.508112 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:00.508071    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:08:00.826312 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:08:00.826231    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601a1248_dcd8_4100_bead_8888cb2b1622.slice/crio-513dce1caa7ef3772673ac598add715f336faae0013ca1fbfb14ccc8589731bf WatchSource:0}: Error finding container 513dce1caa7ef3772673ac598add715f336faae0013ca1fbfb14ccc8589731bf: Status 404 returned error can't find the container with id 513dce1caa7ef3772673ac598add715f336faae0013ca1fbfb14ccc8589731bf
Oct 30 01:08:00.827216 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:00.827155    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"]
Oct 30 01:08:01.439603 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:01.439560    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx" event={"ID":"601a1248-dcd8-4100-bead-8888cb2b1622","Type":"ContainerStarted","Data":"1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933"}
Oct 30 01:08:01.439773 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:01.439633    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx" event={"ID":"601a1248-dcd8-4100-bead-8888cb2b1622","Type":"ContainerStarted","Data":"513dce1caa7ef3772673ac598add715f336faae0013ca1fbfb14ccc8589731bf"}
Oct 30 01:08:01.439773 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:01.439661    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:08:01.484931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:01.484822    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx" podStartSLOduration=2.120445949 podStartE2EDuration="2.484805859s" totalImagesPullingTime="364.35991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:01.483767778 +0000 UTC m=+1723.586521749" watchObservedRunningTime="2026-10-30 01:08:01.484805859 +0000 UTC m=+1723.587559830"
Oct 30 01:08:05.237398 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.237358    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"]
Oct 30 01:08:05.242225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.242200    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.261879 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.261845    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"]
Oct 30 01:08:05.278075 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.278028    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-metrics-certs\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.278311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.278100    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.278311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.278146    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.278311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.278254    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blzh7\" (UniqueName: \"kubernetes.io/projected/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-kube-api-access-blzh7\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.279523 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.279494    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-d54608-serving-cert\""
Oct 30 01:08:05.379702 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.379658    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-metrics-certs\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.379892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.379722    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.379892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.379768    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.379892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.379874    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blzh7\" (UniqueName: \"kubernetes.io/projected/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-kube-api-access-blzh7\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.380013 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:08:05.379893    6011 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-serving-cert: secret "maas-api-e2e-inf-b-d54608-serving-cert" not found
Oct 30 01:08:05.380013 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:08:05.379975    6011 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls podName:c8a84b94-b272-4927-b0cd-4c33ba3bb6d3 nodeName:}" failed. No retries permitted until 2026-10-30 01:08:05.879958081 +0000 UTC m=+1727.982712030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls") pod "maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx" (UID: "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3") : secret "maas-api-e2e-inf-b-d54608-serving-cert" not found
Oct 30 01:08:05.380647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.380582    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.382654 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.382626    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-metrics-certs\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.400777 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.400740    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blzh7\" (UniqueName: \"kubernetes.io/projected/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-kube-api-access-blzh7\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.887249 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.887152    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:05.889825 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:05.889804    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls\") pod \"maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:06.159369 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:06.159284    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:06.475876 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:06.475836    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"]
Oct 30 01:08:06.478983 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:08:06.478929    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8a84b94_b272_4927_b0cd_4c33ba3bb6d3.slice/crio-fd3eeb7db8f57b0602eaeb341e6f0aa6140e40f94dcddd2cdca6482e28f25842 WatchSource:0}: Error finding container fd3eeb7db8f57b0602eaeb341e6f0aa6140e40f94dcddd2cdca6482e28f25842: Status 404 returned error can't find the container with id fd3eeb7db8f57b0602eaeb341e6f0aa6140e40f94dcddd2cdca6482e28f25842
Oct 30 01:08:07.458224 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:07.458191    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:08:07.494042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:07.494002    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx" event={"ID":"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3","Type":"ContainerStarted","Data":"12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3"}
Oct 30 01:08:07.494525 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:07.494074    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:07.494525 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:07.494093    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx" event={"ID":"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3","Type":"ContainerStarted","Data":"fd3eeb7db8f57b0602eaeb341e6f0aa6140e40f94dcddd2cdca6482e28f25842"}
Oct 30 01:08:07.550373 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:07.550299    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx" podStartSLOduration=2.203154024 podStartE2EDuration="2.550276122s" totalImagesPullingTime="347.122098ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:07.544564764 +0000 UTC m=+1729.647318735" watchObservedRunningTime="2026-10-30 01:08:07.550276122 +0000 UTC m=+1729.653030096"
Oct 30 01:08:13.507440 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:13.507408    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:08:27.912054 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:08:27.912008    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:08:47.996224 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:47.996167    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"]
Oct 30 01:08:48.002522 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.002492    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.024217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.024165    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"]
Oct 30 01:08:48.030495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.030458    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1eae0b\"/\"kube-root-ca.crt\""
Oct 30 01:08:48.030718 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.030458    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-1eae0b\"/\"openshift-service-ca.crt\""
Oct 30 01:08:48.032615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.032593    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1eae0b\"/\"default-dockercfg-jmzjl\""
Oct 30 01:08:48.032758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.032646    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-1eae0b\"/\"test-model-1eae0b-kserve-self-signed-certs\""
Oct 30 01:08:48.164711 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.164668    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2de426d-a263-4d98-8e1f-e81dd394db13-tls-certs\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.164909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.164780    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-model-cache\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.164909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.164847    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-tmp-dir\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.164986 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.164970    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6hf7\" (UniqueName: \"kubernetes.io/projected/e2de426d-a263-4d98-8e1f-e81dd394db13-kube-api-access-j6hf7\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.165071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.165057    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-home\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.165159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.165147    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-dshm\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.267144 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.267028    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6hf7\" (UniqueName: \"kubernetes.io/projected/e2de426d-a263-4d98-8e1f-e81dd394db13-kube-api-access-j6hf7\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.267390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.267167    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-home\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.267390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.267288    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-dshm\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.267390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.267339    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2de426d-a263-4d98-8e1f-e81dd394db13-tls-certs\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.267630 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.267393    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-model-cache\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.267630 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.267477    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-tmp-dir\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.267991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.267967    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-tmp-dir\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.268601 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.268573    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-home\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.269532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.269506    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-model-cache\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.277846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.277762    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2de426d-a263-4d98-8e1f-e81dd394db13-tls-certs\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.293623 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.293589    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6hf7\" (UniqueName: \"kubernetes.io/projected/e2de426d-a263-4d98-8e1f-e81dd394db13-kube-api-access-j6hf7\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.294139 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.294105    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-dshm\") pod \"test-model-1eae0b-kserve-667865d579-dgcjv\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") " pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.318298 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.318263    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:48.627349 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:08:48.627314    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode2de426d_a263_4d98_8e1f_e81dd394db13.slice/crio-99af3d9aea5d3abd90e3cefdaa4fabc98080d582aee0be538897e4064dc62b3e WatchSource:0}: Error finding container 99af3d9aea5d3abd90e3cefdaa4fabc98080d582aee0be538897e4064dc62b3e: Status 404 returned error can't find the container with id 99af3d9aea5d3abd90e3cefdaa4fabc98080d582aee0be538897e4064dc62b3e
Oct 30 01:08:48.628406 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.628342    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"]
Oct 30 01:08:48.821965 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.821922    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv" event={"ID":"e2de426d-a263-4d98-8e1f-e81dd394db13","Type":"ContainerStarted","Data":"d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5"}
Oct 30 01:08:48.822134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.821977    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv" event={"ID":"e2de426d-a263-4d98-8e1f-e81dd394db13","Type":"ContainerStarted","Data":"99af3d9aea5d3abd90e3cefdaa4fabc98080d582aee0be538897e4064dc62b3e"}
Oct 30 01:08:48.866282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:48.866222    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv" podStartSLOduration=1.8662077940000001 podStartE2EDuration="1.866207794s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:48.861918467 +0000 UTC m=+1770.964672438" watchObservedRunningTime="2026-10-30 01:08:48.866207794 +0000 UTC m=+1770.968961787"
Oct 30 01:08:49.319286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:49.319246    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:53.470121 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.470084    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"]
Oct 30 01:08:53.474922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.474895    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.500222 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.499655    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8e8ad1\"/\"kube-root-ca.crt\""
Oct 30 01:08:53.505646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.504940    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8e8ad1\"/\"test-model-8e8ad1-kserve-self-signed-certs\""
Oct 30 01:08:53.505646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.505033    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-8e8ad1\"/\"openshift-service-ca.crt\""
Oct 30 01:08:53.505646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.505082    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-8e8ad1\"/\"default-dockercfg-2jgcw\""
Oct 30 01:08:53.507259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.507230    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"]
Oct 30 01:08:53.541815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.541783    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-home\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.541815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.541826    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tmp-dir\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.542057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.541985    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tls-certs\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.542196 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.542155    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-625lp\" (UniqueName: \"kubernetes.io/projected/4455d70b-7e7d-496a-aa21-eeac3a88af1a-kube-api-access-625lp\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.542300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.542284    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-dshm\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.542359 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.542346    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-model-cache\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.643511 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.643469    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-home\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.643511 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.643519    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tmp-dir\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.643792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.643598    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tls-certs\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.643792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.643703    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-625lp\" (UniqueName: \"kubernetes.io/projected/4455d70b-7e7d-496a-aa21-eeac3a88af1a-kube-api-access-625lp\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.643792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.643762    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-dshm\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.643962 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.643800    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-model-cache\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.644022 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.643992    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-home\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.644102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.644072    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tmp-dir\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.644267 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.644238    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-model-cache\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.646202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.646165    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-dshm\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.646436 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.646415    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tls-certs\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.665045 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.665015    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-625lp\" (UniqueName: \"kubernetes.io/projected/4455d70b-7e7d-496a-aa21-eeac3a88af1a-kube-api-access-625lp\") pod \"test-model-8e8ad1-kserve-5cb497c6d9-slphz\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") " pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:53.819256 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:53.811610    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:54.173763 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:08:54.173733    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4455d70b_7e7d_496a_aa21_eeac3a88af1a.slice/crio-6db15e2f66a6a6ca72252b8eca3cf2be9979ccb0b6eedfa05fa46503ce4fd0bf WatchSource:0}: Error finding container 6db15e2f66a6a6ca72252b8eca3cf2be9979ccb0b6eedfa05fa46503ce4fd0bf: Status 404 returned error can't find the container with id 6db15e2f66a6a6ca72252b8eca3cf2be9979ccb0b6eedfa05fa46503ce4fd0bf
Oct 30 01:08:54.174499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:54.174475    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"]
Oct 30 01:08:54.872077 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:54.872040    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz" event={"ID":"4455d70b-7e7d-496a-aa21-eeac3a88af1a","Type":"ContainerStarted","Data":"687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a"}
Oct 30 01:08:54.872493 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:54.872093    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz" event={"ID":"4455d70b-7e7d-496a-aa21-eeac3a88af1a","Type":"ContainerStarted","Data":"6db15e2f66a6a6ca72252b8eca3cf2be9979ccb0b6eedfa05fa46503ce4fd0bf"}
Oct 30 01:08:54.943526 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:54.943472    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz" podStartSLOduration=1.943455729 podStartE2EDuration="1.943455729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:08:54.942134577 +0000 UTC m=+1777.044888548" watchObservedRunningTime="2026-10-30 01:08:54.943455729 +0000 UTC m=+1777.046209699"
Oct 30 01:08:55.813025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:55.812992    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:08:57.913718 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:08:57.913670    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:08:58.318939 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:58.318896    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:58.332229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:58.332200    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:08:58.915125 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:08:58.915095    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:09:03.812980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:03.812944    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:09:03.825835 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:03.825806    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:09:03.961099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:03.961064    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:09:27.912969 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:09:27.912872    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:09:33.855104 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.855070    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"]
Oct 30 01:09:33.861036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.861014    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.880045 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.880017    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"]
Oct 30 01:09:33.891233 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.891196    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b483fd-openshift-default-dockercfg-rdt2m\""
Oct 30 01:09:33.907568 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.907528    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-data\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.907781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.907690    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-token\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.907781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.907722    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-socket\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.907781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.907751    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-credential-socket\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.907951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.907847    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dbcef76-2808-44f5-800a-3bf46f79faaf-istiod-ca-cert\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.907951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.907885    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7kj5\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-kube-api-access-m7kj5\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.908055 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.907994    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-envoy\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.908055 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.908010    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-podinfo\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:33.908055 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:33.908030    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-certs\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.008994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.008952    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-token\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.008994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009000    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-socket\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009124    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-credential-socket\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009242    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dbcef76-2808-44f5-800a-3bf46f79faaf-istiod-ca-cert\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009294    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7kj5\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-kube-api-access-m7kj5\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009383    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-envoy\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009399    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-podinfo\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009432    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-certs\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009432    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-socket\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009459    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-credential-socket\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009711 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009586    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-data\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009769 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009745    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-certs\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.009902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.009881    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-data\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.010157 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.010044    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dbcef76-2808-44f5-800a-3bf46f79faaf-istiod-ca-cert\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.011885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.011859    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-envoy\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.012071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.012052    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-podinfo\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.030630 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.030592    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-token\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.030777 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.030712    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7kj5\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-kube-api-access-m7kj5\") pod \"e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") " pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.176274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.176163    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:34.496074 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.496030    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"]
Oct 30 01:09:34.496480 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:09:34.496451    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbcef76_2808_44f5_800a_3bf46f79faaf.slice/crio-02f2627ca2fb537ac74e8bfc9b9cbfda156ebfdebc3e930d3fcfe47bb7d9ead5 WatchSource:0}: Error finding container 02f2627ca2fb537ac74e8bfc9b9cbfda156ebfdebc3e930d3fcfe47bb7d9ead5: Status 404 returned error can't find the container with id 02f2627ca2fb537ac74e8bfc9b9cbfda156ebfdebc3e930d3fcfe47bb7d9ead5
Oct 30 01:09:34.499487 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.499436    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:09:34.499607 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.499555    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:09:34.499607 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:34.499602    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:09:35.264250 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:35.260962    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" event={"ID":"3dbcef76-2808-44f5-800a-3bf46f79faaf","Type":"ContainerStarted","Data":"8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff"}
Oct 30 01:09:35.264250 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:35.261092    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" event={"ID":"3dbcef76-2808-44f5-800a-3bf46f79faaf","Type":"ContainerStarted","Data":"02f2627ca2fb537ac74e8bfc9b9cbfda156ebfdebc3e930d3fcfe47bb7d9ead5"}
Oct 30 01:09:35.334761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:35.334695    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podStartSLOduration=2.334683125 podStartE2EDuration="2.334683125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:09:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:09:35.317966324 +0000 UTC m=+1817.420720294" watchObservedRunningTime="2026-10-30 01:09:35.334683125 +0000 UTC m=+1817.437437096"
Oct 30 01:09:36.177239 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:36.177161    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:36.179069 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:36.179041    6011 patch_prober.go:29] interesting pod/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:09:36.179255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:36.179092    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:09:37.181502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:37.181439    6011 patch_prober.go:29] interesting pod/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:09:37.182016 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:37.181552    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:09:38.176815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:38.176772    6011 patch_prober.go:29] interesting pod/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:09:38.177036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:38.176857    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:09:39.177114 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:39.177078    6011 patch_prober.go:29] interesting pod/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:09:39.177570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:39.177139    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:09:40.182614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:40.182570    6011 patch_prober.go:29] interesting pod/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused" start-of-body=
Oct 30 01:09:40.183217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:40.183153    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.119:15021/healthz/ready\": dial tcp 10.134.0.119:15021: connect: connection refused"
Oct 30 01:09:41.195471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:41.195435    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:41.195968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:41.195942    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:41.197327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:41.197302    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:09:48.281224 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:48.281190    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"]
Oct 30 01:09:48.281727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:48.281466    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy" containerID="cri-o://c5e9f664fae4b62025a9048022977dbf18cd6b45fe42262525cd4bce19f5b92f" gracePeriod=30
Oct 30 01:09:53.459753 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.456569    6011 generic.go:356] "Generic (PLEG): container finished" podID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerID="c5e9f664fae4b62025a9048022977dbf18cd6b45fe42262525cd4bce19f5b92f" exitCode=0
Oct 30 01:09:53.459753 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.456773    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" event={"ID":"32b4fc90-89b9-427a-8c73-e770d836d92f","Type":"ContainerDied","Data":"c5e9f664fae4b62025a9048022977dbf18cd6b45fe42262525cd4bce19f5b92f"}
Oct 30 01:09:53.618011 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.617982    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 01:09:53.667416 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667379    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-certs\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667590 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667440    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-credential-socket\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667590 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667464    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-socket\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667590 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667490    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-token\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-token\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667590 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667509    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-envoy\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667590 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667529    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-kube-api-access-5vsqv\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-kube-api-access-5vsqv\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667590 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667587    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/32b4fc90-89b9-427a-8c73-e770d836d92f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b4fc90-89b9-427a-8c73-e770d836d92f-istiod-ca-cert\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667897 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667716    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-data\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667897 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667759    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-podinfo\") pod \"32b4fc90-89b9-427a-8c73-e770d836d92f\" (UID: \"32b4fc90-89b9-427a-8c73-e770d836d92f\") "
Oct 30 01:09:53.667897 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667840    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-credential-socket" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:53.668101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.667928    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-socket" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:53.668101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668088    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-data" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:53.668431 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668268    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-certs" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:53.668431 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668300    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32b4fc90-89b9-427a-8c73-e770d836d92f-istiod-ca-cert" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:09:53.668811 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668783    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/32b4fc90-89b9-427a-8c73-e770d836d92f-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.668936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668814    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.668936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668829    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.668936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668843    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.668936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.668858    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.670263 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.670238    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-envoy" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:09:53.670377 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.670350    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-kube-api-access-5vsqv" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "kube-api-access-5vsqv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:53.670807 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.670784    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-token" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:09:53.670892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.670814    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-podinfo" pod "32b4fc90-89b9-427a-8c73-e770d836d92f" (UID: "32b4fc90-89b9-427a-8c73-e770d836d92f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:09:53.771163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.771111    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.771163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.771161    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.771399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.771196    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vsqv\" (UniqueName: \"kubernetes.io/projected/32b4fc90-89b9-427a-8c73-e770d836d92f-kube-api-access-5vsqv\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:53.771399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:53.771212    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/32b4fc90-89b9-427a-8c73-e770d836d92f-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:09:54.466962 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:54.466870    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2" event={"ID":"32b4fc90-89b9-427a-8c73-e770d836d92f","Type":"ContainerDied","Data":"18ac625ec18df232f649555a80b002f9068338e89a581c9008950289cae2948b"}
Oct 30 01:09:54.467466 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:54.466891    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"
Oct 30 01:09:54.467466 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:54.466990    6011 scope.go:122] "RemoveContainer" containerID="c5e9f664fae4b62025a9048022977dbf18cd6b45fe42262525cd4bce19f5b92f"
Oct 30 01:09:54.549368 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:54.549335    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"]
Oct 30 01:09:54.596205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:54.595450    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-pbjt2"]
Oct 30 01:09:55.044402 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:55.044361    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"]
Oct 30 01:09:56.026395 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:09:56.026357    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" path="/var/lib/kubelet/pods/32b4fc90-89b9-427a-8c73-e770d836d92f/volumes"
Oct 30 01:09:57.914308 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:09:57.914269    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:10:10.074523 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.074467    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97" podUID="31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" containerName="main" containerID="cri-o://cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15" gracePeriod=45
Oct 30 01:10:10.143226 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:10:10.143157    6011 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31a3e6d8_409b_41b9_bd30_40dc7fb9d0f2.slice/crio-cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod31a3e6d8_409b_41b9_bd30_40dc7fb9d0f2.slice/crio-conmon-cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 01:10:10.373320 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.373294    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:10:10.446564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.446514    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-home\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-home\") pod \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") "
Oct 30 01:10:10.446767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.446602    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-model-cache\") pod \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") "
Oct 30 01:10:10.446767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.446698    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-dshm\") pod \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") "
Oct 30 01:10:10.446903 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.446820    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tmp-dir\") pod \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") "
Oct 30 01:10:10.446903 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.446860    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-home" pod "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" (UID: "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:10.446903 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.446870    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tls-certs\" (UniqueName: \"kubernetes.io/secret/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tls-certs\") pod \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") "
Oct 30 01:10:10.447064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.446903    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-model-cache" pod "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" (UID: "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:10.447064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.447036    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-kube-api-access-jk9ds\" (UniqueName: \"kubernetes.io/projected/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-kube-api-access-jk9ds\") pod \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\" (UID: \"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2\") "
Oct 30 01:10:10.451234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.447775    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:10.451234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.447796    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:10.451234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.448326    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tmp-dir" pod "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" (UID: "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:10.451234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.449857    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-dshm" pod "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" (UID: "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:10.451234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.449993    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tls-certs" pod "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" (UID: "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:10:10.451234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.450622    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-kube-api-access-jk9ds" pod "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" (UID: "31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2"). InnerVolumeSpecName "kube-api-access-jk9ds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:10.549234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.549155    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:10.549234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.549210    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:10.549234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.549228    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jk9ds\" (UniqueName: \"kubernetes.io/projected/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-kube-api-access-jk9ds\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:10.549234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.549242    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:10.598991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.598892    6011 generic.go:356] "Generic (PLEG): container finished" podID="31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" containerID="cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15" exitCode=0
Oct 30 01:10:10.598991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.598943    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"
Oct 30 01:10:10.598991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.598967    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97" event={"ID":"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2","Type":"ContainerDied","Data":"cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15"}
Oct 30 01:10:10.599284 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.599003    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97" event={"ID":"31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2","Type":"ContainerDied","Data":"68e6ca732da3990b04edfb8d1de920f1d44abe353cb70d56588ce6dd1b0d5dcf"}
Oct 30 01:10:10.599284 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.599022    6011 scope.go:122] "RemoveContainer" containerID="cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15"
Oct 30 01:10:10.611715 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.611690    6011 scope.go:122] "RemoveContainer" containerID="cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15"
Oct 30 01:10:10.612050 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:10:10.612027    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15\": container with ID starting with cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15 not found: ID does not exist" containerID="cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15"
Oct 30 01:10:10.612105 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.612063    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15"} err="failed to get container status \"cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15\": rpc error: code = NotFound desc = could not find container \"cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15\": container with ID starting with cf3b9e6b1d68cdfa2b904fbac34b90c17c373b7af770765fe83019454e1a6b15 not found: ID does not exist"
Oct 30 01:10:10.656867 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.656834    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"]
Oct 30 01:10:10.660689 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:10.660659    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-2ca20f/test-model-2ca20f-kserve-89cf6d47-5sg97"]
Oct 30 01:10:12.022729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:12.022694    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" path="/var/lib/kubelet/pods/31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2/volumes"
Oct 30 01:10:19.175823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:19.175781    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"]
Oct 30 01:10:27.913831 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:10:27.913790    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:10:33.067781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:33.067741    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"]
Oct 30 01:10:33.068225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:33.067951    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy" containerID="cri-o://201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e" gracePeriod=30
Oct 30 01:10:33.703480 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:33.703449    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"]
Oct 30 01:10:33.703680 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:33.703643    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy" containerID="cri-o://717dd57e7a9cfdbe9f281fb6e2916163186408fe9113e6617707a7258b8bcbee" gracePeriod=30
Oct 30 01:10:34.200199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.200146    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f" podUID="5fd3e120-b2be-4ec1-a33d-0564033820b2" containerName="main" containerID="cri-o://afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034" gracePeriod=45
Oct 30 01:10:34.478589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.478555    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:10:34.548741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.548702    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-model-cache\") pod \"5fd3e120-b2be-4ec1-a33d-0564033820b2\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") "
Oct 30 01:10:34.548957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.548761    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5fd3e120-b2be-4ec1-a33d-0564033820b2-kube-api-access-m52ds\" (UniqueName: \"kubernetes.io/projected/5fd3e120-b2be-4ec1-a33d-0564033820b2-kube-api-access-m52ds\") pod \"5fd3e120-b2be-4ec1-a33d-0564033820b2\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") "
Oct 30 01:10:34.548957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.548862    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-tmp-dir\") pod \"5fd3e120-b2be-4ec1-a33d-0564033820b2\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") "
Oct 30 01:10:34.549122 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.549058    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-model-cache" pod "5fd3e120-b2be-4ec1-a33d-0564033820b2" (UID: "5fd3e120-b2be-4ec1-a33d-0564033820b2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:34.549249 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.549155    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-tmp-dir" pod "5fd3e120-b2be-4ec1-a33d-0564033820b2" (UID: "5fd3e120-b2be-4ec1-a33d-0564033820b2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:34.549411 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.549392    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5fd3e120-b2be-4ec1-a33d-0564033820b2-tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd3e120-b2be-4ec1-a33d-0564033820b2-tls-certs\") pod \"5fd3e120-b2be-4ec1-a33d-0564033820b2\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") "
Oct 30 01:10:34.549547 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.549526    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-home\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-home\") pod \"5fd3e120-b2be-4ec1-a33d-0564033820b2\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") "
Oct 30 01:10:34.549840 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.549821    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-dshm\") pod \"5fd3e120-b2be-4ec1-a33d-0564033820b2\" (UID: \"5fd3e120-b2be-4ec1-a33d-0564033820b2\") "
Oct 30 01:10:34.550108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.550080    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-home" pod "5fd3e120-b2be-4ec1-a33d-0564033820b2" (UID: "5fd3e120-b2be-4ec1-a33d-0564033820b2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:34.551277 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.551250    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:34.551383 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.551279    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:34.551383 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.551293    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:34.552355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.552328    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fd3e120-b2be-4ec1-a33d-0564033820b2-kube-api-access-m52ds" pod "5fd3e120-b2be-4ec1-a33d-0564033820b2" (UID: "5fd3e120-b2be-4ec1-a33d-0564033820b2"). InnerVolumeSpecName "kube-api-access-m52ds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:34.552631 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.552604    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fd3e120-b2be-4ec1-a33d-0564033820b2-tls-certs" pod "5fd3e120-b2be-4ec1-a33d-0564033820b2" (UID: "5fd3e120-b2be-4ec1-a33d-0564033820b2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:10:34.553479 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.553455    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-dshm" pod "5fd3e120-b2be-4ec1-a33d-0564033820b2" (UID: "5fd3e120-b2be-4ec1-a33d-0564033820b2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:34.652255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.652214    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5fd3e120-b2be-4ec1-a33d-0564033820b2-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:34.652255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.652242    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5fd3e120-b2be-4ec1-a33d-0564033820b2-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:34.652255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.652254    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m52ds\" (UniqueName: \"kubernetes.io/projected/5fd3e120-b2be-4ec1-a33d-0564033820b2-kube-api-access-m52ds\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:34.832790 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.832756    6011 generic.go:356] "Generic (PLEG): container finished" podID="5fd3e120-b2be-4ec1-a33d-0564033820b2" containerID="afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034" exitCode=0
Oct 30 01:10:34.833003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.832802    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"
Oct 30 01:10:34.833003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.832844    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f" event={"ID":"5fd3e120-b2be-4ec1-a33d-0564033820b2","Type":"ContainerDied","Data":"afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034"}
Oct 30 01:10:34.833003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.832898    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f" event={"ID":"5fd3e120-b2be-4ec1-a33d-0564033820b2","Type":"ContainerDied","Data":"02925eaad58a07400c8ebd56d9d1db4a3709591a52f90257fe7d8e7bc1074ed6"}
Oct 30 01:10:34.833003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.832921    6011 scope.go:122] "RemoveContainer" containerID="afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034"
Oct 30 01:10:34.844436 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.844416    6011 scope.go:122] "RemoveContainer" containerID="afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034"
Oct 30 01:10:34.844739 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:10:34.844718    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034\": container with ID starting with afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034 not found: ID does not exist" containerID="afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034"
Oct 30 01:10:34.844811 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.844745    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034"} err="failed to get container status \"afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034\": rpc error: code = NotFound desc = could not find container \"afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034\": container with ID starting with afa2eb35773a754ae3bdd18b0a19b45b9f050d303e770908b6fcdc16263dc034 not found: ID does not exist"
Oct 30 01:10:34.888500 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.888466    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"]
Oct 30 01:10:34.896827 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:34.896798    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-b483fd/test-model-b483fd-kserve-5bdcf886fc-kqd6f"]
Oct 30 01:10:36.023255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:36.023220    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5fd3e120-b2be-4ec1-a33d-0564033820b2" path="/var/lib/kubelet/pods/5fd3e120-b2be-4ec1-a33d-0564033820b2/volumes"
Oct 30 01:10:38.363915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.362581    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 01:10:38.415307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415263    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-podinfo\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415435    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-socket\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415534    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-certs\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415650 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415608    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-token\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415664    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-credential-socket\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415723    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-kube-api-access-kngcg\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-kube-api-access-kngcg\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415764    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-data\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415851    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-envoy\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415884    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istiod-ca-cert\") pod \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\" (UID: \"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1\") "
Oct 30 01:10:38.415969 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.415935    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-certs" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:38.416165 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.416139    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-socket" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:38.416743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.416521    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istiod-ca-cert" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:38.416880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.416759    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.416880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.416782    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.416880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.416796    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.417083 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.417061    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-data" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:38.420064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.420001    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-credential-socket" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:38.420339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.420312    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-podinfo" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:38.422288 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.422260    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-kube-api-access-kngcg" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "kube-api-access-kngcg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:38.426474 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.425968    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-envoy" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:38.439921 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.439888    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-token" pod "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" (UID: "a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:38.519692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.519628    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.519692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.519671    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.519692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.519688    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kngcg\" (UniqueName: \"kubernetes.io/projected/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-kube-api-access-kngcg\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.519692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.519701    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.520282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.519712    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.520282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.519723    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:38.867522 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.867483    6011 generic.go:356] "Generic (PLEG): container finished" podID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerID="201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e" exitCode=0
Oct 30 01:10:38.867705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.867529    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"
Oct 30 01:10:38.867705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.867560    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" event={"ID":"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1","Type":"ContainerDied","Data":"201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e"}
Oct 30 01:10:38.867705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.867631    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm" event={"ID":"a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1","Type":"ContainerDied","Data":"0143cf01cae0003bd6ad3b433c935779ae5b357d72d35658373d7724d2a0d519"}
Oct 30 01:10:38.867705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.867671    6011 scope.go:122] "RemoveContainer" containerID="201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e"
Oct 30 01:10:38.869568 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.869542    6011 generic.go:356] "Generic (PLEG): container finished" podID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerID="717dd57e7a9cfdbe9f281fb6e2916163186408fe9113e6617707a7258b8bcbee" exitCode=0
Oct 30 01:10:38.869703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.869645    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" event={"ID":"2e2a58fe-562d-4d20-9a12-fa42928bc9d2","Type":"ContainerDied","Data":"717dd57e7a9cfdbe9f281fb6e2916163186408fe9113e6617707a7258b8bcbee"}
Oct 30 01:10:38.881605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.881587    6011 scope.go:122] "RemoveContainer" containerID="201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e"
Oct 30 01:10:38.881921 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:10:38.881901    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e\": container with ID starting with 201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e not found: ID does not exist" containerID="201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e"
Oct 30 01:10:38.881989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.881930    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e"} err="failed to get container status \"201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e\": rpc error: code = NotFound desc = could not find container \"201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e\": container with ID starting with 201634fa8bdcf24e7f79187df82181a72a9e76c985803501e48a17ac74f5530e not found: ID does not exist"
Oct 30 01:10:38.945966 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.945932    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"]
Oct 30 01:10:38.957854 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.957799    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-vnlkm"]
Oct 30 01:10:38.963980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:38.963959    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 01:10:39.027941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.027861    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-data\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.027941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.027918    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-credential-socket\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.028204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028023    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istiod-ca-cert\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.028204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028088    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-podinfo\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.028335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028235    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-data" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:39.028335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028262    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-token\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-token\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.028335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028302    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-envoy\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.028494 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028337    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-credential-socket" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:39.028494 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028389    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-socket\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.028494 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028468    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-kube-api-access-wvk2x\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-kube-api-access-wvk2x\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.028646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028508    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-certs\") pod \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\" (UID: \"2e2a58fe-562d-4d20-9a12-fa42928bc9d2\") "
Oct 30 01:10:39.029351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028759    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-socket" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:39.029351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.028985    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-certs" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:39.029351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.029304    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istiod-ca-cert" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:10:39.029657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.029437    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.029657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.029456    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.029657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.029473    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.029657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.029487    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.029657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.029501    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.031089 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.031063    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-envoy" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:10:39.031350 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.031323    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-token" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:39.031350 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.031341    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-kube-api-access-wvk2x" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "kube-api-access-wvk2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:10:39.031948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.031924    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-podinfo" pod "2e2a58fe-562d-4d20-9a12-fa42928bc9d2" (UID: "2e2a58fe-562d-4d20-9a12-fa42928bc9d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:10:39.131265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.131232    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.131265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.131259    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.131265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.131270    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvk2x\" (UniqueName: \"kubernetes.io/projected/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-kube-api-access-wvk2x\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.131265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.131279    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2e2a58fe-562d-4d20-9a12-fa42928bc9d2-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:10:39.879618 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.879584    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"
Oct 30 01:10:39.880108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.879616    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7" event={"ID":"2e2a58fe-562d-4d20-9a12-fa42928bc9d2","Type":"ContainerDied","Data":"e26e106e1f2472851fd5935079a5eebb554ea6e6dcfc7e9f09786991f73da50f"}
Oct 30 01:10:39.880108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.879679    6011 scope.go:122] "RemoveContainer" containerID="717dd57e7a9cfdbe9f281fb6e2916163186408fe9113e6617707a7258b8bcbee"
Oct 30 01:10:39.990605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.990562    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"]
Oct 30 01:10:39.994790 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:39.994753    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-bh7d7"]
Oct 30 01:10:40.023263 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:40.023230    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" path="/var/lib/kubelet/pods/2e2a58fe-562d-4d20-9a12-fa42928bc9d2/volumes"
Oct 30 01:10:40.023815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:10:40.023800    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" path="/var/lib/kubelet/pods/a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1/volumes"
Oct 30 01:10:57.912143 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:10:57.912096    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:11:17.816858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:17.816815    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"]
Oct 30 01:11:17.817414 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:17.817155    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy" containerID="cri-o://333b210dd1c1831fcd62326dd0b2ad65e031b54b08125dd66111d8be214eef15" gracePeriod=30
Oct 30 01:11:18.707861 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:18.707814    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"]
Oct 30 01:11:18.708074 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:18.708049    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy" containerID="cri-o://a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092" gracePeriod=30
Oct 30 01:11:19.611541 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:19.611506    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"]
Oct 30 01:11:19.612065 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:19.611728    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy" containerID="cri-o://2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f" gracePeriod=30
Oct 30 01:11:23.234686 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.234639    6011 generic.go:356] "Generic (PLEG): container finished" podID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerID="333b210dd1c1831fcd62326dd0b2ad65e031b54b08125dd66111d8be214eef15" exitCode=0
Oct 30 01:11:23.235053 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.234711    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" event={"ID":"9a1a771f-88ba-49b1-89fa-630dee6e723f","Type":"ContainerDied","Data":"333b210dd1c1831fcd62326dd0b2ad65e031b54b08125dd66111d8be214eef15"}
Oct 30 01:11:23.258645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.258620    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 01:11:23.305704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.305666    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a1a771f-88ba-49b1-89fa-630dee6e723f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a1a771f-88ba-49b1-89fa-630dee6e723f-istiod-ca-cert\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.305886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.305730    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-data\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.305886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.305842    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-credential-socket\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.305886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.305871    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-certs\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.306042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.305904    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-podinfo\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.306042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306008    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-envoy\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.306161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306060    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-data" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:23.306161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306106    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-kube-api-access-vbszb\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-kube-api-access-vbszb\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.306161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306067    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a1a771f-88ba-49b1-89fa-630dee6e723f-istiod-ca-cert" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:11:23.306161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306139    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-token\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-token\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.306400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306211    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-credential-socket" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:23.306400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306231    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-certs" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:23.306400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306256    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-socket\") pod \"9a1a771f-88ba-49b1-89fa-630dee6e723f\" (UID: \"9a1a771f-88ba-49b1-89fa-630dee6e723f\") "
Oct 30 01:11:23.306553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306510    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-socket" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:23.306885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306864    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.306885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306886    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.307057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306899    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.307057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306911    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a1a771f-88ba-49b1-89fa-630dee6e723f-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.307057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.306924    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.308546 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.308512    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-envoy" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:23.308546 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.308524    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-token" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:23.308830 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.308806    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-kube-api-access-vbszb" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "kube-api-access-vbszb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:23.308830 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.308819    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-podinfo" pod "9a1a771f-88ba-49b1-89fa-630dee6e723f" (UID: "9a1a771f-88ba-49b1-89fa-630dee6e723f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:11:23.408040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.408005    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.408040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.408038    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.408281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.408049    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbszb\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-kube-api-access-vbszb\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.408281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.408060    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a1a771f-88ba-49b1-89fa-630dee6e723f-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:23.992208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:23.992115    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 01:11:24.117562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117516    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-kube-api-access-52z2h\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-kube-api-access-52z2h\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.117741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117586    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-data\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.117741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117664    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-credential-socket\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.117741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117696    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b145eea6-be64-417d-8c27-104f040067ee-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b145eea6-be64-417d-8c27-104f040067ee-istio-podinfo\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.117927 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117772    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-istio-token\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-istio-token\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.117927 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117823    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-certs\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.117927 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117884    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b145eea6-be64-417d-8c27-104f040067ee-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b145eea6-be64-417d-8c27-104f040067ee-istiod-ca-cert\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.118080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117981    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-socket\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.118080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.117986    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-data" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:24.118080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118012    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-credential-socket" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:24.118301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118123    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-certs" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:24.118301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118130    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-envoy\") pod \"b145eea6-be64-417d-8c27-104f040067ee\" (UID: \"b145eea6-be64-417d-8c27-104f040067ee\") "
Oct 30 01:11:24.118301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118250    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-socket" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:24.118423 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118362    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b145eea6-be64-417d-8c27-104f040067ee-istiod-ca-cert" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:11:24.118872 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118819    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b145eea6-be64-417d-8c27-104f040067ee-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.118872 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118842    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.118872 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118851    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.118872 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118859    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.118872 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.118868    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.120150 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.120121    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b145eea6-be64-417d-8c27-104f040067ee-istio-podinfo" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:11:24.120448 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.120426    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-envoy" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:24.120586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.120569    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-kube-api-access-52z2h" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "kube-api-access-52z2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:24.120637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.120582    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-istio-token" pod "b145eea6-be64-417d-8c27-104f040067ee" (UID: "b145eea6-be64-417d-8c27-104f040067ee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:24.220352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.220307    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.220352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.220348    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b145eea6-be64-417d-8c27-104f040067ee-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.220352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.220358    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52z2h\" (UniqueName: \"kubernetes.io/projected/b145eea6-be64-417d-8c27-104f040067ee-kube-api-access-52z2h\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.220352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.220370    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b145eea6-be64-417d-8c27-104f040067ee-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:24.243771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.243680    6011 generic.go:356] "Generic (PLEG): container finished" podID="b145eea6-be64-417d-8c27-104f040067ee" containerID="a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092" exitCode=0
Oct 30 01:11:24.243771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.243730    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"
Oct 30 01:11:24.244318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.243767    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" event={"ID":"b145eea6-be64-417d-8c27-104f040067ee","Type":"ContainerDied","Data":"a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092"}
Oct 30 01:11:24.244318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.243824    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg" event={"ID":"b145eea6-be64-417d-8c27-104f040067ee","Type":"ContainerDied","Data":"ba545b892204ede91ce2bf2e9beada017d4c5840892b4f9957370d398da22dea"}
Oct 30 01:11:24.244318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.243855    6011 scope.go:122] "RemoveContainer" containerID="a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092"
Oct 30 01:11:24.245475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.245451    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm" event={"ID":"9a1a771f-88ba-49b1-89fa-630dee6e723f","Type":"ContainerDied","Data":"59812e46cb592897fc4517e4a8c235e1408528a8fe5342d7cc7b91263f2aaa5f"}
Oct 30 01:11:24.245582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.245489    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"
Oct 30 01:11:24.260316 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.260294    6011 scope.go:122] "RemoveContainer" containerID="a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092"
Oct 30 01:11:24.260618 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:11:24.260601    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092\": container with ID starting with a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092 not found: ID does not exist" containerID="a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092"
Oct 30 01:11:24.260671 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.260626    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092"} err="failed to get container status \"a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092\": rpc error: code = NotFound desc = could not find container \"a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092\": container with ID starting with a5ca9aaf31d0c78f7d752c17bb2335c441a2504b073e65c0faa5a7b44c203092 not found: ID does not exist"
Oct 30 01:11:24.260671 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.260642    6011 scope.go:122] "RemoveContainer" containerID="333b210dd1c1831fcd62326dd0b2ad65e031b54b08125dd66111d8be214eef15"
Oct 30 01:11:24.300687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.300657    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"]
Oct 30 01:11:24.305502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.305476    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-wx6mg"]
Oct 30 01:11:24.342371 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.342340    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"]
Oct 30 01:11:24.345585 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.345563    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-7d2b75-openshift-default-7d976f567-8nsxm"]
Oct 30 01:11:24.897327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:24.897257    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 01:11:25.032046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032009    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-data\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032045    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-envoy\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032068    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a97b2579-278d-4aa6-b930-9172b08163a2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a97b2579-278d-4aa6-b930-9172b08163a2-istiod-ca-cert\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032217    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-certs\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032266    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-kube-api-access-k6jgx\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-kube-api-access-k6jgx\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032345    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-data" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:25.032475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032416    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-credential-socket\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032491    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a97b2579-278d-4aa6-b930-9172b08163a2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a97b2579-278d-4aa6-b930-9172b08163a2-istio-podinfo\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032554    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-certs" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:25.032589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032561    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a97b2579-278d-4aa6-b930-9172b08163a2-istiod-ca-cert" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:11:25.032589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032579    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-socket\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032615    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-istio-token\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-istio-token\") pod \"a97b2579-278d-4aa6-b930-9172b08163a2\" (UID: \"a97b2579-278d-4aa6-b930-9172b08163a2\") "
Oct 30 01:11:25.032798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032701    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-credential-socket" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:25.032885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.032794    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-socket" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:25.033437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.033410    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.033579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.033440    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.033579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.033455    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a97b2579-278d-4aa6-b930-9172b08163a2-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.033579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.033468    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.033579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.033481    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.034758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.034734    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-envoy" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:25.034758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.034748    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a97b2579-278d-4aa6-b930-9172b08163a2-istio-podinfo" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:11:25.034884 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.034776    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-kube-api-access-k6jgx" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "kube-api-access-k6jgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:25.035158 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.035129    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-istio-token" pod "a97b2579-278d-4aa6-b930-9172b08163a2" (UID: "a97b2579-278d-4aa6-b930-9172b08163a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:25.135245 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.135209    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.135245 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.135245    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a97b2579-278d-4aa6-b930-9172b08163a2-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.135572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.135260    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6jgx\" (UniqueName: \"kubernetes.io/projected/a97b2579-278d-4aa6-b930-9172b08163a2-kube-api-access-k6jgx\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.135572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.135274    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a97b2579-278d-4aa6-b930-9172b08163a2-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:25.255297 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.255208    6011 generic.go:356] "Generic (PLEG): container finished" podID="a97b2579-278d-4aa6-b930-9172b08163a2" containerID="2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f" exitCode=0
Oct 30 01:11:25.255297 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.255256    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"
Oct 30 01:11:25.255800 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.255300    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" event={"ID":"a97b2579-278d-4aa6-b930-9172b08163a2","Type":"ContainerDied","Data":"2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f"}
Oct 30 01:11:25.255800 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.255359    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t" event={"ID":"a97b2579-278d-4aa6-b930-9172b08163a2","Type":"ContainerDied","Data":"c9e6a1894fc8bbd77b1088bc551e32bcf7f22e8d25f7f6d47a50fec324f844db"}
Oct 30 01:11:25.255800 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.255394    6011 scope.go:122] "RemoveContainer" containerID="2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f"
Oct 30 01:11:25.267022 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.267004    6011 scope.go:122] "RemoveContainer" containerID="2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f"
Oct 30 01:11:25.267365 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:11:25.267344    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f\": container with ID starting with 2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f not found: ID does not exist" containerID="2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f"
Oct 30 01:11:25.267440 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.267375    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f"} err="failed to get container status \"2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f\": rpc error: code = NotFound desc = could not find container \"2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f\": container with ID starting with 2c42fa1e6174e0857d41163aadeb5f7479a742c45514b3ddd8bf14ac9f77f41f not found: ID does not exist"
Oct 30 01:11:25.313041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.312998    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"]
Oct 30 01:11:25.323167 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:25.323135    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-wl67t"]
Oct 30 01:11:26.022647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:26.022614    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" path="/var/lib/kubelet/pods/9a1a771f-88ba-49b1-89fa-630dee6e723f/volumes"
Oct 30 01:11:26.023203 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:26.023160    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" path="/var/lib/kubelet/pods/a97b2579-278d-4aa6-b930-9172b08163a2/volumes"
Oct 30 01:11:26.023636 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:26.023622    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b145eea6-be64-417d-8c27-104f040067ee" path="/var/lib/kubelet/pods/b145eea6-be64-417d-8c27-104f040067ee/volumes"
Oct 30 01:11:27.912334 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:11:27.912295    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:11:40.600922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:40.600884    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"]
Oct 30 01:11:48.896629 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:48.896594    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"]
Oct 30 01:11:52.672468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:52.672427    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"]
Oct 30 01:11:52.673032 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:52.672697    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78" podUID="89d25b2a-f431-41a4-912b-8b5bdc763a54" containerName="maas-api" containerID="cri-o://b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb" gracePeriod=30
Oct 30 01:11:53.187450 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.187414    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:11:53.337727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.337673    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-metrics-certs\") pod \"89d25b2a-f431-41a4-912b-8b5bdc763a54\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") "
Oct 30 01:11:53.337913 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.337741    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/89d25b2a-f431-41a4-912b-8b5bdc763a54-kube-api-access-hg56r\" (UniqueName: \"kubernetes.io/projected/89d25b2a-f431-41a4-912b-8b5bdc763a54-kube-api-access-hg56r\") pod \"89d25b2a-f431-41a4-912b-8b5bdc763a54\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") "
Oct 30 01:11:53.337999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.337969    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-maas-api-tls\") pod \"89d25b2a-f431-41a4-912b-8b5bdc763a54\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") "
Oct 30 01:11:53.338061 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.338007    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/89d25b2a-f431-41a4-912b-8b5bdc763a54-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d25b2a-f431-41a4-912b-8b5bdc763a54-openshift-service-ca\") pod \"89d25b2a-f431-41a4-912b-8b5bdc763a54\" (UID: \"89d25b2a-f431-41a4-912b-8b5bdc763a54\") "
Oct 30 01:11:53.338523 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.338439    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89d25b2a-f431-41a4-912b-8b5bdc763a54-openshift-service-ca" pod "89d25b2a-f431-41a4-912b-8b5bdc763a54" (UID: "89d25b2a-f431-41a4-912b-8b5bdc763a54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:11:53.338523 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.338555    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/89d25b2a-f431-41a4-912b-8b5bdc763a54-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:53.340274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.340247    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-metrics-certs" pod "89d25b2a-f431-41a4-912b-8b5bdc763a54" (UID: "89d25b2a-f431-41a4-912b-8b5bdc763a54"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:11:53.340476 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.340450    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-maas-api-tls" pod "89d25b2a-f431-41a4-912b-8b5bdc763a54" (UID: "89d25b2a-f431-41a4-912b-8b5bdc763a54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:11:53.340556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.340452    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89d25b2a-f431-41a4-912b-8b5bdc763a54-kube-api-access-hg56r" pod "89d25b2a-f431-41a4-912b-8b5bdc763a54" (UID: "89d25b2a-f431-41a4-912b-8b5bdc763a54"). InnerVolumeSpecName "kube-api-access-hg56r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:53.439565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.439529    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:53.439565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.439560    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/89d25b2a-f431-41a4-912b-8b5bdc763a54-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:53.439565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.439570    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg56r\" (UniqueName: \"kubernetes.io/projected/89d25b2a-f431-41a4-912b-8b5bdc763a54-kube-api-access-hg56r\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:53.480565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.480517    6011 generic.go:356] "Generic (PLEG): container finished" podID="89d25b2a-f431-41a4-912b-8b5bdc763a54" containerID="b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb" exitCode=0
Oct 30 01:11:53.480767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.480581    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"
Oct 30 01:11:53.480767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.480606    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78" event={"ID":"89d25b2a-f431-41a4-912b-8b5bdc763a54","Type":"ContainerDied","Data":"b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb"}
Oct 30 01:11:53.480767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.480669    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78" event={"ID":"89d25b2a-f431-41a4-912b-8b5bdc763a54","Type":"ContainerDied","Data":"79a3faa33782bff34c60c50eb200de0b62113e6f480399dd24d842c5bdaba390"}
Oct 30 01:11:53.480767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.480697    6011 scope.go:122] "RemoveContainer" containerID="b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb"
Oct 30 01:11:53.500416 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.500390    6011 scope.go:122] "RemoveContainer" containerID="b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb"
Oct 30 01:11:53.500814 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:11:53.500792    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb\": container with ID starting with b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb not found: ID does not exist" containerID="b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb"
Oct 30 01:11:53.500881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.500824    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb"} err="failed to get container status \"b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb\": rpc error: code = NotFound desc = could not find container \"b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb\": container with ID starting with b10054daacd6577d16f83ee89f0d8026adefcb3f54736c02a463a0a6ca68d4eb not found: ID does not exist"
Oct 30 01:11:53.537073 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.537033    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"]
Oct 30 01:11:53.551860 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:53.551817    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-49d9f1-668b9d5ffb-jjp78"]
Oct 30 01:11:54.022943 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:54.022909    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="89d25b2a-f431-41a4-912b-8b5bdc763a54" path="/var/lib/kubelet/pods/89d25b2a-f431-41a4-912b-8b5bdc763a54/volumes"
Oct 30 01:11:55.623286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.623217    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv" podUID="e2de426d-a263-4d98-8e1f-e81dd394db13" containerName="main" containerID="cri-o://d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5" gracePeriod=45
Oct 30 01:11:55.913316 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.913288    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:11:55.974615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.974568    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2de426d-a263-4d98-8e1f-e81dd394db13-kube-api-access-j6hf7\" (UniqueName: \"kubernetes.io/projected/e2de426d-a263-4d98-8e1f-e81dd394db13-kube-api-access-j6hf7\") pod \"e2de426d-a263-4d98-8e1f-e81dd394db13\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") "
Oct 30 01:11:55.974827 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.974631    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-dshm\") pod \"e2de426d-a263-4d98-8e1f-e81dd394db13\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") "
Oct 30 01:11:55.974827 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.974676    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e2de426d-a263-4d98-8e1f-e81dd394db13-tls-certs\" (UniqueName: \"kubernetes.io/secret/e2de426d-a263-4d98-8e1f-e81dd394db13-tls-certs\") pod \"e2de426d-a263-4d98-8e1f-e81dd394db13\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") "
Oct 30 01:11:55.974827 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.974710    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-tmp-dir\") pod \"e2de426d-a263-4d98-8e1f-e81dd394db13\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") "
Oct 30 01:11:55.974827 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.974743    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-model-cache\") pod \"e2de426d-a263-4d98-8e1f-e81dd394db13\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") "
Oct 30 01:11:55.975068 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.974905    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-home\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-home\") pod \"e2de426d-a263-4d98-8e1f-e81dd394db13\" (UID: \"e2de426d-a263-4d98-8e1f-e81dd394db13\") "
Oct 30 01:11:55.975137 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.975051    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-tmp-dir" pod "e2de426d-a263-4d98-8e1f-e81dd394db13" (UID: "e2de426d-a263-4d98-8e1f-e81dd394db13"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:55.975137 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.975075    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-model-cache" pod "e2de426d-a263-4d98-8e1f-e81dd394db13" (UID: "e2de426d-a263-4d98-8e1f-e81dd394db13"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:55.975375 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.975303    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-home" pod "e2de426d-a263-4d98-8e1f-e81dd394db13" (UID: "e2de426d-a263-4d98-8e1f-e81dd394db13"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:55.975549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.975532    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:55.975615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.975549    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:55.975615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.975560    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:55.977739 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.977710    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2de426d-a263-4d98-8e1f-e81dd394db13-tls-certs" pod "e2de426d-a263-4d98-8e1f-e81dd394db13" (UID: "e2de426d-a263-4d98-8e1f-e81dd394db13"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:11:55.977981 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.977957    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2de426d-a263-4d98-8e1f-e81dd394db13-kube-api-access-j6hf7" pod "e2de426d-a263-4d98-8e1f-e81dd394db13" (UID: "e2de426d-a263-4d98-8e1f-e81dd394db13"). InnerVolumeSpecName "kube-api-access-j6hf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:11:55.978277 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:55.978257    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-dshm" pod "e2de426d-a263-4d98-8e1f-e81dd394db13" (UID: "e2de426d-a263-4d98-8e1f-e81dd394db13"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:11:56.076647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.076605    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6hf7\" (UniqueName: \"kubernetes.io/projected/e2de426d-a263-4d98-8e1f-e81dd394db13-kube-api-access-j6hf7\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:56.076647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.076647    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e2de426d-a263-4d98-8e1f-e81dd394db13-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:56.076927 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.076664    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e2de426d-a263-4d98-8e1f-e81dd394db13-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:11:56.504833 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.504796    6011 generic.go:356] "Generic (PLEG): container finished" podID="e2de426d-a263-4d98-8e1f-e81dd394db13" containerID="d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5" exitCode=0
Oct 30 01:11:56.505063 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.504865    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"
Oct 30 01:11:56.505063 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.504864    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv" event={"ID":"e2de426d-a263-4d98-8e1f-e81dd394db13","Type":"ContainerDied","Data":"d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5"}
Oct 30 01:11:56.505063 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.504989    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv" event={"ID":"e2de426d-a263-4d98-8e1f-e81dd394db13","Type":"ContainerDied","Data":"99af3d9aea5d3abd90e3cefdaa4fabc98080d582aee0be538897e4064dc62b3e"}
Oct 30 01:11:56.505063 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.505014    6011 scope.go:122] "RemoveContainer" containerID="d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5"
Oct 30 01:11:56.516207 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.516153    6011 scope.go:122] "RemoveContainer" containerID="d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5"
Oct 30 01:11:56.516578 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:11:56.516555    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5\": container with ID starting with d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5 not found: ID does not exist" containerID="d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5"
Oct 30 01:11:56.516636 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.516589    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5"} err="failed to get container status \"d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5\": rpc error: code = NotFound desc = could not find container \"d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5\": container with ID starting with d16beb3edd6f94e0ff0c7843015e0605147069e077e64fabdc05f4917e75cec5 not found: ID does not exist"
Oct 30 01:11:56.560883 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.560848    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"]
Oct 30 01:11:56.573565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.573518    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv"]
Oct 30 01:11:56.903997 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:56.903937    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-1eae0b/test-model-1eae0b-kserve-667865d579-dgcjv" podUID="e2de426d-a263-4d98-8e1f-e81dd394db13" containerName="main" probeResult="failure" output="Get \"https://10.134.0.117:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:11:57.917398 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:11:57.917270    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:11:58.023295 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:58.023258    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2de426d-a263-4d98-8e1f-e81dd394db13" path="/var/lib/kubelet/pods/e2de426d-a263-4d98-8e1f-e81dd394db13/volumes"
Oct 30 01:11:59.847689 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:59.847650    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"]
Oct 30 01:11:59.848204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:11:59.847926    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd" podUID="ceeb5930-31fd-4f81-9edb-93423e0f621f" containerName="maas-api" containerID="cri-o://c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063" gracePeriod=30
Oct 30 01:12:00.128025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.127993    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:12:00.230241 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.230202    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-maas-api-tls\") pod \"ceeb5930-31fd-4f81-9edb-93423e0f621f\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") "
Oct 30 01:12:00.230439 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.230424    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-metrics-certs\") pod \"ceeb5930-31fd-4f81-9edb-93423e0f621f\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") "
Oct 30 01:12:00.230574 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.230552    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ceeb5930-31fd-4f81-9edb-93423e0f621f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceeb5930-31fd-4f81-9edb-93423e0f621f-openshift-service-ca\") pod \"ceeb5930-31fd-4f81-9edb-93423e0f621f\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") "
Oct 30 01:12:00.230856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.230612    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ceeb5930-31fd-4f81-9edb-93423e0f621f-kube-api-access-8ng4h\" (UniqueName: \"kubernetes.io/projected/ceeb5930-31fd-4f81-9edb-93423e0f621f-kube-api-access-8ng4h\") pod \"ceeb5930-31fd-4f81-9edb-93423e0f621f\" (UID: \"ceeb5930-31fd-4f81-9edb-93423e0f621f\") "
Oct 30 01:12:00.231245 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.231221    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceeb5930-31fd-4f81-9edb-93423e0f621f-openshift-service-ca" pod "ceeb5930-31fd-4f81-9edb-93423e0f621f" (UID: "ceeb5930-31fd-4f81-9edb-93423e0f621f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:00.233364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.233321    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-metrics-certs" pod "ceeb5930-31fd-4f81-9edb-93423e0f621f" (UID: "ceeb5930-31fd-4f81-9edb-93423e0f621f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:00.233477 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.233410    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceeb5930-31fd-4f81-9edb-93423e0f621f-kube-api-access-8ng4h" pod "ceeb5930-31fd-4f81-9edb-93423e0f621f" (UID: "ceeb5930-31fd-4f81-9edb-93423e0f621f"). InnerVolumeSpecName "kube-api-access-8ng4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:00.233477 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.233418    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-maas-api-tls" pod "ceeb5930-31fd-4f81-9edb-93423e0f621f" (UID: "ceeb5930-31fd-4f81-9edb-93423e0f621f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:00.332499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.332450    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:00.332499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.332487    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ceeb5930-31fd-4f81-9edb-93423e0f621f-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:00.332499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.332499    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ceeb5930-31fd-4f81-9edb-93423e0f621f-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:00.332499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.332508    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ng4h\" (UniqueName: \"kubernetes.io/projected/ceeb5930-31fd-4f81-9edb-93423e0f621f-kube-api-access-8ng4h\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:00.536604 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.536568    6011 generic.go:356] "Generic (PLEG): container finished" podID="ceeb5930-31fd-4f81-9edb-93423e0f621f" containerID="c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063" exitCode=0
Oct 30 01:12:00.536788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.536611    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"
Oct 30 01:12:00.536788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.536646    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd" event={"ID":"ceeb5930-31fd-4f81-9edb-93423e0f621f","Type":"ContainerDied","Data":"c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063"}
Oct 30 01:12:00.536788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.536702    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd" event={"ID":"ceeb5930-31fd-4f81-9edb-93423e0f621f","Type":"ContainerDied","Data":"10752b0be531427a25d9dd3d871f81ad2ba5483ff964d7a0acb92b64b66c3ac5"}
Oct 30 01:12:00.536788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.536725    6011 scope.go:122] "RemoveContainer" containerID="c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063"
Oct 30 01:12:00.550593 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.550568    6011 scope.go:122] "RemoveContainer" containerID="c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063"
Oct 30 01:12:00.550969 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:12:00.550950    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063\": container with ID starting with c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063 not found: ID does not exist" containerID="c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063"
Oct 30 01:12:00.551040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.550978    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063"} err="failed to get container status \"c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063\": rpc error: code = NotFound desc = could not find container \"c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063\": container with ID starting with c22dac554d94fc4d338ab8ad0ac684882ca50843d0b7506a0079da131064d063 not found: ID does not exist"
Oct 30 01:12:00.599163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.599107    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"]
Oct 30 01:12:00.621887 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:00.621846    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b483fd-6b9b47fbc-z7rvd"]
Oct 30 01:12:02.022587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.022551    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ceeb5930-31fd-4f81-9edb-93423e0f621f" path="/var/lib/kubelet/pods/ceeb5930-31fd-4f81-9edb-93423e0f621f/volumes"
Oct 30 01:12:02.935729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.935695    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"]
Oct 30 01:12:02.936395 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936379    6011 cpu_manager.go:414] "containerMap: removing container" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy"
Oct 30 01:12:02.936455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936396    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy"
Oct 30 01:12:02.936455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936410    6011 cpu_manager.go:414] "containerMap: removing container" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy"
Oct 30 01:12:02.936455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936415    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy"
Oct 30 01:12:02.936455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936426    6011 cpu_manager.go:414] "containerMap: removing container" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy"
Oct 30 01:12:02.936455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936431    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy"
Oct 30 01:12:02.936455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936445    6011 cpu_manager.go:414] "containerMap: removing container" podUID="89d25b2a-f431-41a4-912b-8b5bdc763a54" containerName="maas-api"
Oct 30 01:12:02.936455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936451    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="89d25b2a-f431-41a4-912b-8b5bdc763a54" containerName="maas-api"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936477    6011 cpu_manager.go:414] "containerMap: removing container" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936482    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936493    6011 cpu_manager.go:414] "containerMap: removing container" podUID="5fd3e120-b2be-4ec1-a33d-0564033820b2" containerName="main"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936498    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5fd3e120-b2be-4ec1-a33d-0564033820b2" containerName="main"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936513    6011 cpu_manager.go:414] "containerMap: removing container" podUID="ceeb5930-31fd-4f81-9edb-93423e0f621f" containerName="maas-api"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936518    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ceeb5930-31fd-4f81-9edb-93423e0f621f" containerName="maas-api"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936533    6011 cpu_manager.go:414] "containerMap: removing container" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936540    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936549    6011 cpu_manager.go:414] "containerMap: removing container" podUID="e2de426d-a263-4d98-8e1f-e81dd394db13" containerName="main"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936554    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2de426d-a263-4d98-8e1f-e81dd394db13" containerName="main"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936566    6011 cpu_manager.go:414] "containerMap: removing container" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936570    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936586    6011 cpu_manager.go:414] "containerMap: removing container" podUID="31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" containerName="main"
Oct 30 01:12:02.936667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936590    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" containerName="main"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936683    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="ceeb5930-31fd-4f81-9edb-93423e0f621f" containerName="maas-api"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936694    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="a6e1d9ee-b865-4b21-a0ac-3f73bc0c6ba1" containerName="istio-proxy"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936700    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2de426d-a263-4d98-8e1f-e81dd394db13" containerName="main"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936707    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="b145eea6-be64-417d-8c27-104f040067ee" containerName="istio-proxy"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936715    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="31a3e6d8-409b-41b9-bd30-40dc7fb9d0f2" containerName="main"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936721    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="32b4fc90-89b9-427a-8c73-e770d836d92f" containerName="istio-proxy"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936726    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="a97b2579-278d-4aa6-b930-9172b08163a2" containerName="istio-proxy"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936733    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="2e2a58fe-562d-4d20-9a12-fa42928bc9d2" containerName="istio-proxy"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936739    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="89d25b2a-f431-41a4-912b-8b5bdc763a54" containerName="maas-api"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936745    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a1a771f-88ba-49b1-89fa-630dee6e723f" containerName="istio-proxy"
Oct 30 01:12:02.937126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.936751    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="5fd3e120-b2be-4ec1-a33d-0564033820b2" containerName="main"
Oct 30 01:12:02.941851 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.941833    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:02.959633 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.959594    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"]
Oct 30 01:12:02.968756 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:02.968722    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-95fe33-serving-cert\""
Oct 30 01:12:03.068835 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.068778    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-maas-api-tls\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.069337 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.068993    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.069337 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.069202    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtrs6\" (UniqueName: \"kubernetes.io/projected/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-kube-api-access-rtrs6\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.069556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.069532    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-metrics-certs\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.171836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.171782    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-metrics-certs\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.172036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.171960    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-maas-api-tls\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.172106 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.172005    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.173484 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.172244    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtrs6\" (UniqueName: \"kubernetes.io/projected/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-kube-api-access-rtrs6\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.173484 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.173130    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.174989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.174960    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-maas-api-tls\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.175152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.175118    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-metrics-certs\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.196451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.196360    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtrs6\" (UniqueName: \"kubernetes.io/projected/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-kube-api-access-rtrs6\") pod \"maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.260712 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.260663    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:03.570603 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:12:03.570573    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7c8458a_e080_49aa_b8ac_1cc2fab48bc0.slice/crio-c265bf98086903fe36ad5b5c747b095c37c4b85261d273e737334983246cc928 WatchSource:0}: Error finding container c265bf98086903fe36ad5b5c747b095c37c4b85261d273e737334983246cc928: Status 404 returned error can't find the container with id c265bf98086903fe36ad5b5c747b095c37c4b85261d273e737334983246cc928
Oct 30 01:12:03.570771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.570748    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"]
Oct 30 01:12:03.572025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.572005    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:12:03.918720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.918676    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz" podUID="4455d70b-7e7d-496a-aa21-eeac3a88af1a" containerName="main" containerID="cri-o://687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a" gracePeriod=45
Oct 30 01:12:03.949919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:03.949878    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz" podUID="4455d70b-7e7d-496a-aa21-eeac3a88af1a" containerName="main" probeResult="failure" output="Get \"https://10.134.0.118:8000/ready\": dial tcp 10.134.0.118:8000: connect: connection refused"
Oct 30 01:12:04.229930 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.229896    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:12:04.288552 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.288511    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4455d70b-7e7d-496a-aa21-eeac3a88af1a-kube-api-access-625lp\" (UniqueName: \"kubernetes.io/projected/4455d70b-7e7d-496a-aa21-eeac3a88af1a-kube-api-access-625lp\") pod \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") "
Oct 30 01:12:04.288823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.288619    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tls-certs\" (UniqueName: \"kubernetes.io/secret/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tls-certs\") pod \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") "
Oct 30 01:12:04.288823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.288671    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tmp-dir\") pod \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") "
Oct 30 01:12:04.289020 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.288989    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tmp-dir" pod "4455d70b-7e7d-496a-aa21-eeac3a88af1a" (UID: "4455d70b-7e7d-496a-aa21-eeac3a88af1a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:04.289307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.289160    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-home\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-home\") pod \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") "
Oct 30 01:12:04.289572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.289458    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-home" pod "4455d70b-7e7d-496a-aa21-eeac3a88af1a" (UID: "4455d70b-7e7d-496a-aa21-eeac3a88af1a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:04.289572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.289528    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-model-cache\") pod \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") "
Oct 30 01:12:04.289759 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.289736    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-model-cache" pod "4455d70b-7e7d-496a-aa21-eeac3a88af1a" (UID: "4455d70b-7e7d-496a-aa21-eeac3a88af1a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:04.289817 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.289764    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-dshm\") pod \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\" (UID: \"4455d70b-7e7d-496a-aa21-eeac3a88af1a\") "
Oct 30 01:12:04.290557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.290531    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:04.290667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.290562    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:04.290667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.290580    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:04.294287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.294255    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tls-certs" pod "4455d70b-7e7d-496a-aa21-eeac3a88af1a" (UID: "4455d70b-7e7d-496a-aa21-eeac3a88af1a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:04.294414 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.294393    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4455d70b-7e7d-496a-aa21-eeac3a88af1a-kube-api-access-625lp" pod "4455d70b-7e7d-496a-aa21-eeac3a88af1a" (UID: "4455d70b-7e7d-496a-aa21-eeac3a88af1a"). InnerVolumeSpecName "kube-api-access-625lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:04.294555 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.294530    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-dshm" pod "4455d70b-7e7d-496a-aa21-eeac3a88af1a" (UID: "4455d70b-7e7d-496a-aa21-eeac3a88af1a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:04.391704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.391645    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4455d70b-7e7d-496a-aa21-eeac3a88af1a-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:04.391704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.391679    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-625lp\" (UniqueName: \"kubernetes.io/projected/4455d70b-7e7d-496a-aa21-eeac3a88af1a-kube-api-access-625lp\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:04.391704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.391694    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4455d70b-7e7d-496a-aa21-eeac3a88af1a-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:04.567217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.567161    6011 generic.go:356] "Generic (PLEG): container finished" podID="4455d70b-7e7d-496a-aa21-eeac3a88af1a" containerID="687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a" exitCode=0
Oct 30 01:12:04.567217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.567207    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz" event={"ID":"4455d70b-7e7d-496a-aa21-eeac3a88af1a","Type":"ContainerDied","Data":"687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a"}
Oct 30 01:12:04.567502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.567232    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"
Oct 30 01:12:04.567502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.567265    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz" event={"ID":"4455d70b-7e7d-496a-aa21-eeac3a88af1a","Type":"ContainerDied","Data":"6db15e2f66a6a6ca72252b8eca3cf2be9979ccb0b6eedfa05fa46503ce4fd0bf"}
Oct 30 01:12:04.567502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.567290    6011 scope.go:122] "RemoveContainer" containerID="687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a"
Oct 30 01:12:04.569141 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.569114    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn" event={"ID":"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0","Type":"ContainerStarted","Data":"4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd"}
Oct 30 01:12:04.569286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.569208    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn" event={"ID":"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0","Type":"ContainerStarted","Data":"c265bf98086903fe36ad5b5c747b095c37c4b85261d273e737334983246cc928"}
Oct 30 01:12:04.569286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.569240    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:04.578559 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.578539    6011 scope.go:122] "RemoveContainer" containerID="687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a"
Oct 30 01:12:04.578848 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:12:04.578828    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a\": container with ID starting with 687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a not found: ID does not exist" containerID="687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a"
Oct 30 01:12:04.578895 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.578858    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a"} err="failed to get container status \"687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a\": rpc error: code = NotFound desc = could not find container \"687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a\": container with ID starting with 687894583ce0f461c027cdc299392be08c7221e02b8c5eb49bd34a7cd7fd675a not found: ID does not exist"
Oct 30 01:12:04.627345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.627270    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn" podStartSLOduration=2.335880746 podStartE2EDuration="2.627162501s" totalImagesPullingTime="291.281755ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:12:04.625377888 +0000 UTC m=+1966.728131858" watchObservedRunningTime="2026-10-30 01:12:04.627162501 +0000 UTC m=+1966.729916483"
Oct 30 01:12:04.664032 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.663996    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"]
Oct 30 01:12:04.667087 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:04.667055    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-8e8ad1/test-model-8e8ad1-kserve-5cb497c6d9-slphz"]
Oct 30 01:12:05.042252 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.042219    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"]
Oct 30 01:12:05.042887 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.042873    6011 cpu_manager.go:414] "containerMap: removing container" podUID="4455d70b-7e7d-496a-aa21-eeac3a88af1a" containerName="main"
Oct 30 01:12:05.042942 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.042889    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4455d70b-7e7d-496a-aa21-eeac3a88af1a" containerName="main"
Oct 30 01:12:05.043031 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.043021    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="4455d70b-7e7d-496a-aa21-eeac3a88af1a" containerName="main"
Oct 30 01:12:05.048030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.048009    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.064994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.064960    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"]
Oct 30 01:12:05.074896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.074865    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-d0ae0d-openshift-default-dockercfg-2cc5d\""
Oct 30 01:12:05.100485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.100367    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a922739c-1386-4b81-86f9-4cd0bc393387-istiod-ca-cert\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.100485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.100432    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-credential-socket\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.100754 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.100603    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-istio-token\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.100754 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.100673    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a922739c-1386-4b81-86f9-4cd0bc393387-istio-podinfo\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.100893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.100875    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-certs\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.101039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.101001    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-envoy\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.101254 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.101229    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-data\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.101369 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.101271    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98krq\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-kube-api-access-98krq\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.101369 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.101320    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-socket\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.202737 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.202684    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a922739c-1386-4b81-86f9-4cd0bc393387-istiod-ca-cert\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.202737 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.202725    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-credential-socket\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.202997 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.202770    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-istio-token\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.202997 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.202796    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a922739c-1386-4b81-86f9-4cd0bc393387-istio-podinfo\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.202997 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.202929    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-certs\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.202997 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.202979    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-envoy\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203089    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-data\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203118    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98krq\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-kube-api-access-98krq\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203163    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-socket\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203335    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-certs\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203579    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-socket\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203763    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a922739c-1386-4b81-86f9-4cd0bc393387-istiod-ca-cert\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203803    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-credential-socket\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.204017 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.203855    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-data\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.205677 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.205643    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-envoy\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.205830 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.205807    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a922739c-1386-4b81-86f9-4cd0bc393387-istio-podinfo\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.223421 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.223387    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-istio-token\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.223653 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.223552    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98krq\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-kube-api-access-98krq\") pod \"e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") " pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.364215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.364067    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:05.702458 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.702426    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"]
Oct 30 01:12:05.702672 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:12:05.702647    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda922739c_1386_4b81_86f9_4cd0bc393387.slice/crio-a1999b30361888d296cff30fd7b77dbc97821b11cffe8f69330d4ffecc77cbd0 WatchSource:0}: Error finding container a1999b30361888d296cff30fd7b77dbc97821b11cffe8f69330d4ffecc77cbd0: Status 404 returned error can't find the container with id a1999b30361888d296cff30fd7b77dbc97821b11cffe8f69330d4ffecc77cbd0
Oct 30 01:12:05.705593 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.705557    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:12:05.705664 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.705634    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:12:05.705697 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:05.705664    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:12:06.025975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:06.025877    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4455d70b-7e7d-496a-aa21-eeac3a88af1a" path="/var/lib/kubelet/pods/4455d70b-7e7d-496a-aa21-eeac3a88af1a/volumes"
Oct 30 01:12:06.586701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:06.586664    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" event={"ID":"a922739c-1386-4b81-86f9-4cd0bc393387","Type":"ContainerStarted","Data":"1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f"}
Oct 30 01:12:06.587143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:06.586726    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" event={"ID":"a922739c-1386-4b81-86f9-4cd0bc393387","Type":"ContainerStarted","Data":"a1999b30361888d296cff30fd7b77dbc97821b11cffe8f69330d4ffecc77cbd0"}
Oct 30 01:12:06.650660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:06.650588    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" podStartSLOduration=1.650498499 podStartE2EDuration="1.650498499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:12:06.645564562 +0000 UTC m=+1968.748318537" watchObservedRunningTime="2026-10-30 01:12:06.650498499 +0000 UTC m=+1968.753252471"
Oct 30 01:12:07.364447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:07.364399    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:07.366261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:07.366229    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:12:07.366385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:07.366281    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:12:07.691549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:07.691457    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"]
Oct 30 01:12:07.692015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:07.691737    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy" containerID="cri-o://8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff" gracePeriod=30
Oct 30 01:12:08.364906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:08.364466    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:12:08.364906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:08.364528    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:12:09.365580 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:09.365539    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:12:09.366011 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:09.365614    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:12:10.364672 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:10.364624    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:12:10.364957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:10.364698    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:12:10.583788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:10.583750    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:12:11.198746 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:11.198707    6011 patch_prober.go:29] interesting pod/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:12:11.198931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:11.198770    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:12:11.364775 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:11.364731    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused" start-of-body=
Oct 30 01:12:11.364983 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:11.364804    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.121:15021/healthz/ready\": dial tcp 10.134.0.121:15021: connect: connection refused"
Oct 30 01:12:12.621820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:12.618986    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:12.621820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:12.619602    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:12.621820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:12.621530    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:12:12.981789 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:12.981758    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:12:13.062194 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062146    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3dbcef76-2808-44f5-800a-3bf46f79faaf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dbcef76-2808-44f5-800a-3bf46f79faaf-istiod-ca-cert\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.062386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062265    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-data\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.062386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062327    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-envoy\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.062386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062372    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-token\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-token\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.062678 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062645    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-data" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:13.062778 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062667    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3dbcef76-2808-44f5-800a-3bf46f79faaf-istiod-ca-cert" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:13.062855 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062430    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-certs\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.062981 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.062962    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-podinfo\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.063084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.063059    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-certs" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:13.063679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.063222    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-socket\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.063679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.063281    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-kube-api-access-m7kj5\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-kube-api-access-m7kj5\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.063679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.063315    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-credential-socket\") pod \"3dbcef76-2808-44f5-800a-3bf46f79faaf\" (UID: \"3dbcef76-2808-44f5-800a-3bf46f79faaf\") "
Oct 30 01:12:13.063679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.063645    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-socket" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:13.064030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.063834    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-credential-socket" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:13.064706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.064612    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.064706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.064644    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.064706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.064658    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3dbcef76-2808-44f5-800a-3bf46f79faaf-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.064706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.064672    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.064706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.064683    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.065642 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.065618    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-token" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:13.065924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.065906    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-envoy" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:12:13.066118 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.066098    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-podinfo" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:12:13.067988 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.067969    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-kube-api-access-m7kj5" pod "3dbcef76-2808-44f5-800a-3bf46f79faaf" (UID: "3dbcef76-2808-44f5-800a-3bf46f79faaf"). InnerVolumeSpecName "kube-api-access-m7kj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:13.176039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.170498    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.176039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.170543    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.176039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.170582    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3dbcef76-2808-44f5-800a-3bf46f79faaf-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.176039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.170598    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7kj5\" (UniqueName: \"kubernetes.io/projected/3dbcef76-2808-44f5-800a-3bf46f79faaf-kube-api-access-m7kj5\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:13.659544 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.659506    6011 generic.go:356] "Generic (PLEG): container finished" podID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerID="8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff" exitCode=0
Oct 30 01:12:13.659976 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.659560    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"
Oct 30 01:12:13.659976 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.659588    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" event={"ID":"3dbcef76-2808-44f5-800a-3bf46f79faaf","Type":"ContainerDied","Data":"8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff"}
Oct 30 01:12:13.659976 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.659648    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2" event={"ID":"3dbcef76-2808-44f5-800a-3bf46f79faaf","Type":"ContainerDied","Data":"02f2627ca2fb537ac74e8bfc9b9cbfda156ebfdebc3e930d3fcfe47bb7d9ead5"}
Oct 30 01:12:13.659976 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.659678    6011 scope.go:122] "RemoveContainer" containerID="8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff"
Oct 30 01:12:13.671810 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.671786    6011 scope.go:122] "RemoveContainer" containerID="8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff"
Oct 30 01:12:13.672100 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:12:13.672080    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff\": container with ID starting with 8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff not found: ID does not exist" containerID="8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff"
Oct 30 01:12:13.672205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.672109    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff"} err="failed to get container status \"8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff\": rpc error: code = NotFound desc = could not find container \"8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff\": container with ID starting with 8b4c51698656283c5542396ca905a6aee8365c29ea3f886d1e4feee6830b77ff not found: ID does not exist"
Oct 30 01:12:13.728285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.728248    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"]
Oct 30 01:12:13.741589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:13.741558    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b483fd-openshift-default-64bb6d775-hs8l2"]
Oct 30 01:12:14.023516 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.023430    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" path="/var/lib/kubelet/pods/3dbcef76-2808-44f5-800a-3bf46f79faaf/volumes"
Oct 30 01:12:14.500949 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.500916    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"]
Oct 30 01:12:14.502001 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.501976    6011 cpu_manager.go:414] "containerMap: removing container" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy"
Oct 30 01:12:14.502101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.502003    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy"
Oct 30 01:12:14.502249 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.502234    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="3dbcef76-2808-44f5-800a-3bf46f79faaf" containerName="istio-proxy"
Oct 30 01:12:14.507588 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.507567    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.520824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.520789    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"]
Oct 30 01:12:14.529909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.529882    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"openshift-service-ca.crt\""
Oct 30 01:12:14.530047 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.529961    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"default-dockercfg-r6w79\""
Oct 30 01:12:14.536734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.536706    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"kube-root-ca.crt\""
Oct 30 01:12:14.536862 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.536788    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"auth-test-model-283eb6-kserve-self-signed-certs\""
Oct 30 01:12:14.594225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.594149    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-tmp-dir\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.594415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.594323    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-home\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.594415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.594374    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-dshm\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.594533 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.594510    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2615321f-445d-4aa1-85f7-cb1a6fb94207-tls-certs\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.594575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.594562    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-model-cache\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.594651 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.594633    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv6gs\" (UniqueName: \"kubernetes.io/projected/2615321f-445d-4aa1-85f7-cb1a6fb94207-kube-api-access-fv6gs\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.695723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.695682    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-tmp-dir\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.695746    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-home\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.695785    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-dshm\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.695864    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2615321f-445d-4aa1-85f7-cb1a6fb94207-tls-certs\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.695914    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-model-cache\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.695941    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv6gs\" (UniqueName: \"kubernetes.io/projected/2615321f-445d-4aa1-85f7-cb1a6fb94207-kube-api-access-fv6gs\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.696224    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-tmp-dir\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.696357    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-home\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.696605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.696432    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-model-cache\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.698466 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.698449    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-dshm\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.698766 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.698747    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2615321f-445d-4aa1-85f7-cb1a6fb94207-tls-certs\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.717019 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.716986    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv6gs\" (UniqueName: \"kubernetes.io/projected/2615321f-445d-4aa1-85f7-cb1a6fb94207-kube-api-access-fv6gs\") pod \"auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:14.828757 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:14.828720    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:15.135357 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:15.135323    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"]
Oct 30 01:12:15.135489 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:12:15.135440    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2615321f_445d_4aa1_85f7_cb1a6fb94207.slice/crio-7c14dc3ce4f9729f491680b79032395960ed3741be67d58434ffdf48e8b8058f WatchSource:0}: Error finding container 7c14dc3ce4f9729f491680b79032395960ed3741be67d58434ffdf48e8b8058f: Status 404 returned error can't find the container with id 7c14dc3ce4f9729f491680b79032395960ed3741be67d58434ffdf48e8b8058f
Oct 30 01:12:15.676373 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:15.676333    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr" event={"ID":"2615321f-445d-4aa1-85f7-cb1a6fb94207","Type":"ContainerStarted","Data":"20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704"}
Oct 30 01:12:15.676373 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:15.676376    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr" event={"ID":"2615321f-445d-4aa1-85f7-cb1a6fb94207","Type":"ContainerStarted","Data":"7c14dc3ce4f9729f491680b79032395960ed3741be67d58434ffdf48e8b8058f"}
Oct 30 01:12:15.718061 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:15.718015    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr" podStartSLOduration=1.71800086 podStartE2EDuration="1.71800086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:12:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:12:15.716153271 +0000 UTC m=+1977.818907242" watchObservedRunningTime="2026-10-30 01:12:15.71800086 +0000 UTC m=+1977.820754830"
Oct 30 01:12:15.829698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:15.829655    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:24.131659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.131616    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"]
Oct 30 01:12:24.132352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.131863    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz" podUID="0d58fbab-07ee-4078-a279-4494d6479e33" containerName="maas-api" containerID="cri-o://9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e" gracePeriod=30
Oct 30 01:12:24.479632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.479591    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:12:24.546206 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.546133    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d58fbab-07ee-4078-a279-4494d6479e33-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d58fbab-07ee-4078-a279-4494d6479e33-openshift-service-ca\") pod \"0d58fbab-07ee-4078-a279-4494d6479e33\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") "
Oct 30 01:12:24.546459 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.546268    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d58fbab-07ee-4078-a279-4494d6479e33-kube-api-access-4ffct\" (UniqueName: \"kubernetes.io/projected/0d58fbab-07ee-4078-a279-4494d6479e33-kube-api-access-4ffct\") pod \"0d58fbab-07ee-4078-a279-4494d6479e33\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") "
Oct 30 01:12:24.546459 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.546308    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-metrics-certs\") pod \"0d58fbab-07ee-4078-a279-4494d6479e33\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") "
Oct 30 01:12:24.546459 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.546428    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-maas-api-tls\") pod \"0d58fbab-07ee-4078-a279-4494d6479e33\" (UID: \"0d58fbab-07ee-4078-a279-4494d6479e33\") "
Oct 30 01:12:24.547141 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.547084    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d58fbab-07ee-4078-a279-4494d6479e33-openshift-service-ca" pod "0d58fbab-07ee-4078-a279-4494d6479e33" (UID: "0d58fbab-07ee-4078-a279-4494d6479e33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:24.551853 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.551811    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-maas-api-tls" pod "0d58fbab-07ee-4078-a279-4494d6479e33" (UID: "0d58fbab-07ee-4078-a279-4494d6479e33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:24.555487 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.554745    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d58fbab-07ee-4078-a279-4494d6479e33-kube-api-access-4ffct" pod "0d58fbab-07ee-4078-a279-4494d6479e33" (UID: "0d58fbab-07ee-4078-a279-4494d6479e33"). InnerVolumeSpecName "kube-api-access-4ffct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:24.557693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.557659    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-metrics-certs" pod "0d58fbab-07ee-4078-a279-4494d6479e33" (UID: "0d58fbab-07ee-4078-a279-4494d6479e33"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:24.648786 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.648743    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0d58fbab-07ee-4078-a279-4494d6479e33-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.648786 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.648784    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ffct\" (UniqueName: \"kubernetes.io/projected/0d58fbab-07ee-4078-a279-4494d6479e33-kube-api-access-4ffct\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.648786 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.648795    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.649121 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.648804    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0d58fbab-07ee-4078-a279-4494d6479e33-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:24.756763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.756641    6011 generic.go:356] "Generic (PLEG): container finished" podID="0d58fbab-07ee-4078-a279-4494d6479e33" containerID="9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e" exitCode=0
Oct 30 01:12:24.756763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.756730    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"
Oct 30 01:12:24.756973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.756722    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz" event={"ID":"0d58fbab-07ee-4078-a279-4494d6479e33","Type":"ContainerDied","Data":"9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e"}
Oct 30 01:12:24.756973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.756867    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz" event={"ID":"0d58fbab-07ee-4078-a279-4494d6479e33","Type":"ContainerDied","Data":"e7264bf3c3af56622f549d6d663bd10e09540445f61053b73b9d2c179e834b37"}
Oct 30 01:12:24.756973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.756896    6011 scope.go:122] "RemoveContainer" containerID="9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e"
Oct 30 01:12:24.770590 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.770479    6011 scope.go:122] "RemoveContainer" containerID="9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e"
Oct 30 01:12:24.770951 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:12:24.770928    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e\": container with ID starting with 9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e not found: ID does not exist" containerID="9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e"
Oct 30 01:12:24.771059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.770961    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e"} err="failed to get container status \"9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e\": rpc error: code = NotFound desc = could not find container \"9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e\": container with ID starting with 9d32cd058672430394cf3b3e67a6bee21565d5e329f692b630aa365bfd20da6e not found: ID does not exist"
Oct 30 01:12:24.813584 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.813549    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"]
Oct 30 01:12:24.818226 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.818194    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-8e8ad1-67f456f4b-ftflz"]
Oct 30 01:12:24.828923 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.828889    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:24.842659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:24.842628    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:25.777311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:25.777275    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:12:26.023037 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:26.022996    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d58fbab-07ee-4078-a279-4494d6479e33" path="/var/lib/kubelet/pods/0d58fbab-07ee-4078-a279-4494d6479e33/volumes"
Oct 30 01:12:27.910769 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:12:27.910672    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:12:49.744521 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:49.744473    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"]
Oct 30 01:12:49.745064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:49.744743    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm" podUID="065bcf9d-e7a8-46a7-b730-366f7856f4e4" containerName="maas-api" containerID="cri-o://3c470600e5b6b10fa629ec58858c162d72693f377c3c925cfdd8cd72a8faada4" gracePeriod=30
Oct 30 01:12:49.958209 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:49.958147    6011 generic.go:356] "Generic (PLEG): container finished" podID="065bcf9d-e7a8-46a7-b730-366f7856f4e4" containerID="3c470600e5b6b10fa629ec58858c162d72693f377c3c925cfdd8cd72a8faada4" exitCode=0
Oct 30 01:12:49.958384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:49.958238    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm" event={"ID":"065bcf9d-e7a8-46a7-b730-366f7856f4e4","Type":"ContainerDied","Data":"3c470600e5b6b10fa629ec58858c162d72693f377c3c925cfdd8cd72a8faada4"}
Oct 30 01:12:50.074843 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.074812    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:12:50.222580 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.222536    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/065bcf9d-e7a8-46a7-b730-366f7856f4e4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/065bcf9d-e7a8-46a7-b730-366f7856f4e4-openshift-service-ca\") pod \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") "
Oct 30 01:12:50.222792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.222708    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-metrics-certs\") pod \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") "
Oct 30 01:12:50.222792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.222745    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-maas-api-tls\") pod \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") "
Oct 30 01:12:50.222909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.222864    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/065bcf9d-e7a8-46a7-b730-366f7856f4e4-kube-api-access-gfmrm\" (UniqueName: \"kubernetes.io/projected/065bcf9d-e7a8-46a7-b730-366f7856f4e4-kube-api-access-gfmrm\") pod \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\" (UID: \"065bcf9d-e7a8-46a7-b730-366f7856f4e4\") "
Oct 30 01:12:50.222968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.222908    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/065bcf9d-e7a8-46a7-b730-366f7856f4e4-openshift-service-ca" pod "065bcf9d-e7a8-46a7-b730-366f7856f4e4" (UID: "065bcf9d-e7a8-46a7-b730-366f7856f4e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:12:50.223436 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.223414    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/065bcf9d-e7a8-46a7-b730-366f7856f4e4-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:50.225009 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.224975    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-maas-api-tls" pod "065bcf9d-e7a8-46a7-b730-366f7856f4e4" (UID: "065bcf9d-e7a8-46a7-b730-366f7856f4e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:50.225280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.225054    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/065bcf9d-e7a8-46a7-b730-366f7856f4e4-kube-api-access-gfmrm" pod "065bcf9d-e7a8-46a7-b730-366f7856f4e4" (UID: "065bcf9d-e7a8-46a7-b730-366f7856f4e4"). InnerVolumeSpecName "kube-api-access-gfmrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:12:50.225280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.225093    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-metrics-certs" pod "065bcf9d-e7a8-46a7-b730-366f7856f4e4" (UID: "065bcf9d-e7a8-46a7-b730-366f7856f4e4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:12:50.324585 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.324479    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfmrm\" (UniqueName: \"kubernetes.io/projected/065bcf9d-e7a8-46a7-b730-366f7856f4e4-kube-api-access-gfmrm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:50.324585 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.324512    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:50.324585 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.324522    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/065bcf9d-e7a8-46a7-b730-366f7856f4e4-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:12:50.967554 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.967516    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"
Oct 30 01:12:50.967554 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.967529    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm" event={"ID":"065bcf9d-e7a8-46a7-b730-366f7856f4e4","Type":"ContainerDied","Data":"bac5eb6dfbb63e31ad35fd809d12619629287ebd5c4bf6e583bb0cea212aa28f"}
Oct 30 01:12:50.968099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:50.967594    6011 scope.go:122] "RemoveContainer" containerID="3c470600e5b6b10fa629ec58858c162d72693f377c3c925cfdd8cd72a8faada4"
Oct 30 01:12:51.022721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:51.022687    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"]
Oct 30 01:12:51.035834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:51.035799    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-1eae0b-68f458ffb-hz4mm"]
Oct 30 01:12:52.021848 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:12:52.021813    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="065bcf9d-e7a8-46a7-b730-366f7856f4e4" path="/var/lib/kubelet/pods/065bcf9d-e7a8-46a7-b730-366f7856f4e4/volumes"
Oct 30 01:12:57.915103 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:12:57.915059    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:13:05.478815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.478770    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"]
Oct 30 01:13:05.480345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.479812    6011 cpu_manager.go:414] "containerMap: removing container" podUID="065bcf9d-e7a8-46a7-b730-366f7856f4e4" containerName="maas-api"
Oct 30 01:13:05.480345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.479841    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="065bcf9d-e7a8-46a7-b730-366f7856f4e4" containerName="maas-api"
Oct 30 01:13:05.480345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.479870    6011 cpu_manager.go:414] "containerMap: removing container" podUID="0d58fbab-07ee-4078-a279-4494d6479e33" containerName="maas-api"
Oct 30 01:13:05.480345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.479893    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d58fbab-07ee-4078-a279-4494d6479e33" containerName="maas-api"
Oct 30 01:13:05.480345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.480077    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="065bcf9d-e7a8-46a7-b730-366f7856f4e4" containerName="maas-api"
Oct 30 01:13:05.480345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.480091    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d58fbab-07ee-4078-a279-4494d6479e33" containerName="maas-api"
Oct 30 01:13:05.490485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.490456    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:05.524331 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.524222    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-24d5ce\""
Oct 30 01:13:05.540281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.536915    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-24d5ce-dockercfg-ccv92\""
Oct 30 01:13:05.540281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.540029    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cffdfd57-7d46-40f0-89c3-5d74d036b541-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:05.540281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.540096    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8vs4w\" (UniqueName: \"kubernetes.io/projected/cffdfd57-7d46-40f0-89c3-5d74d036b541-kube-api-access-8vs4w\") pod \"payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:05.577675 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.577636    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"]
Oct 30 01:13:05.641821 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.641776    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cffdfd57-7d46-40f0-89c3-5d74d036b541-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:05.641999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.641859    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8vs4w\" (UniqueName: \"kubernetes.io/projected/cffdfd57-7d46-40f0-89c3-5d74d036b541-kube-api-access-8vs4w\") pod \"payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:05.642586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.642563    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cffdfd57-7d46-40f0-89c3-5d74d036b541-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:05.663278 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.663240    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8vs4w\" (UniqueName: \"kubernetes.io/projected/cffdfd57-7d46-40f0-89c3-5d74d036b541-kube-api-access-8vs4w\") pod \"payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:05.849377 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:05.849341    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:06.152394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.152362    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"]
Oct 30 01:13:06.153014 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:13:06.152989    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcffdfd57_7d46_40f0_89c3_5d74d036b541.slice/crio-8aebbd8d1ab4a951f0011ea8c08b9ff72040c76fa1bce9bce6540f1c44de26fa WatchSource:0}: Error finding container 8aebbd8d1ab4a951f0011ea8c08b9ff72040c76fa1bce9bce6540f1c44de26fa: Status 404 returned error can't find the container with id 8aebbd8d1ab4a951f0011ea8c08b9ff72040c76fa1bce9bce6540f1c44de26fa
Oct 30 01:13:06.537134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.537102    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"]
Oct 30 01:13:06.541795 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.541769    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.558729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.558693    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"]
Oct 30 01:13:06.567163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.567114    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w0-43d152\"/\"openshift-service-ca.crt\""
Oct 30 01:13:06.567458 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.567428    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w0-43d152\"/\"kube-root-ca.crt\""
Oct 30 01:13:06.567594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.567571    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w0-43d152\"/\"auth-test-model-43d152-kserve-self-signed-certs\""
Oct 30 01:13:06.571417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.571391    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w0-43d152\"/\"default-dockercfg-gk4jd\""
Oct 30 01:13:06.654873 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.654832    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-dshm\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.655071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.654895    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgrxh\" (UniqueName: \"kubernetes.io/projected/14f9d6a2-1416-4314-9882-94e33cb9ef8b-kube-api-access-rgrxh\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.655071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.654977    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tls-certs\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.655208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.655166    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-home\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.655341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.655212    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-model-cache\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.655387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.655360    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tmp-dir\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757147 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757109    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-dshm\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757369 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757197    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgrxh\" (UniqueName: \"kubernetes.io/projected/14f9d6a2-1416-4314-9882-94e33cb9ef8b-kube-api-access-rgrxh\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757369 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757227    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tls-certs\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757369 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757353    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-home\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757376    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-model-cache\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757493    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tmp-dir\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757874 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757856    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-home\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.757937 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.757901    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tmp-dir\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.758159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.758142    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-model-cache\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.760234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.760162    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-dshm\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.760944 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.760912    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tls-certs\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.782473 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.782441    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgrxh\" (UniqueName: \"kubernetes.io/projected/14f9d6a2-1416-4314-9882-94e33cb9ef8b-kube-api-access-rgrxh\") pod \"auth-test-model-43d152-kserve-6454d9fd49-qnj29\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") " pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:06.858137 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:06.858002    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:07.110811 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:07.110713    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" event={"ID":"cffdfd57-7d46-40f0-89c3-5d74d036b541","Type":"ContainerStarted","Data":"85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b"}
Oct 30 01:13:07.110811 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:07.110785    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" event={"ID":"cffdfd57-7d46-40f0-89c3-5d74d036b541","Type":"ContainerStarted","Data":"8aebbd8d1ab4a951f0011ea8c08b9ff72040c76fa1bce9bce6540f1c44de26fa"}
Oct 30 01:13:07.111078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:07.111053    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:07.154856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:07.154797    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" podStartSLOduration=2.154781908 podStartE2EDuration="2.154781908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:13:07.149996031 +0000 UTC m=+2029.252750006" watchObservedRunningTime="2026-10-30 01:13:07.154781908 +0000 UTC m=+2029.257535878"
Oct 30 01:13:07.176887 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:07.176854    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"]
Oct 30 01:13:07.177014 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:13:07.176906    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14f9d6a2_1416_4314_9882_94e33cb9ef8b.slice/crio-1eba9f3964f602afacc4c9a403f6ea84100c1e46326934e75a7ae9482f5ca3d4 WatchSource:0}: Error finding container 1eba9f3964f602afacc4c9a403f6ea84100c1e46326934e75a7ae9482f5ca3d4: Status 404 returned error can't find the container with id 1eba9f3964f602afacc4c9a403f6ea84100c1e46326934e75a7ae9482f5ca3d4
Oct 30 01:13:08.119833 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:08.119787    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29" event={"ID":"14f9d6a2-1416-4314-9882-94e33cb9ef8b","Type":"ContainerStarted","Data":"706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee"}
Oct 30 01:13:08.119833 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:08.119835    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29" event={"ID":"14f9d6a2-1416-4314-9882-94e33cb9ef8b","Type":"ContainerStarted","Data":"1eba9f3964f602afacc4c9a403f6ea84100c1e46326934e75a7ae9482f5ca3d4"}
Oct 30 01:13:08.164080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:08.164026    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29" podStartSLOduration=2.164012982 podStartE2EDuration="2.164012982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:13:08.163345448 +0000 UTC m=+2030.266099419" watchObservedRunningTime="2026-10-30 01:13:08.164012982 +0000 UTC m=+2030.266766952"
Oct 30 01:13:08.858637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:08.858600    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:10.784562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:10.784387    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"]
Oct 30 01:13:10.784973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:10.784626    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx" podUID="601a1248-dcd8-4100-bead-8888cb2b1622" containerName="maas-api" containerID="cri-o://1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933" gracePeriod=30
Oct 30 01:13:11.090758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.090726    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:13:11.147816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.147771    6011 generic.go:356] "Generic (PLEG): container finished" podID="601a1248-dcd8-4100-bead-8888cb2b1622" containerID="1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933" exitCode=0
Oct 30 01:13:11.148035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.147850    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"
Oct 30 01:13:11.148035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.147865    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx" event={"ID":"601a1248-dcd8-4100-bead-8888cb2b1622","Type":"ContainerDied","Data":"1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933"}
Oct 30 01:13:11.148035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.147973    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx" event={"ID":"601a1248-dcd8-4100-bead-8888cb2b1622","Type":"ContainerDied","Data":"513dce1caa7ef3772673ac598add715f336faae0013ca1fbfb14ccc8589731bf"}
Oct 30 01:13:11.148035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.148014    6011 scope.go:122] "RemoveContainer" containerID="1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933"
Oct 30 01:13:11.159376 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.159331    6011 scope.go:122] "RemoveContainer" containerID="1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933"
Oct 30 01:13:11.159722 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:13:11.159699    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933\": container with ID starting with 1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933 not found: ID does not exist" containerID="1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933"
Oct 30 01:13:11.159810 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.159736    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933"} err="failed to get container status \"1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933\": rpc error: code = NotFound desc = could not find container \"1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933\": container with ID starting with 1c261182ad6f0e8278dd63f07c032ed1624458194592428afbb7b954d1a35933 not found: ID does not exist"
Oct 30 01:13:11.225128 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.225085    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/601a1248-dcd8-4100-bead-8888cb2b1622-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/601a1248-dcd8-4100-bead-8888cb2b1622-openshift-service-ca\") pod \"601a1248-dcd8-4100-bead-8888cb2b1622\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") "
Oct 30 01:13:11.225361 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.225154    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/601a1248-dcd8-4100-bead-8888cb2b1622-kube-api-access-tdj6d\" (UniqueName: \"kubernetes.io/projected/601a1248-dcd8-4100-bead-8888cb2b1622-kube-api-access-tdj6d\") pod \"601a1248-dcd8-4100-bead-8888cb2b1622\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") "
Oct 30 01:13:11.225361 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.225232    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls\") pod \"601a1248-dcd8-4100-bead-8888cb2b1622\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") "
Oct 30 01:13:11.225487 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.225374    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-metrics-certs\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-metrics-certs\") pod \"601a1248-dcd8-4100-bead-8888cb2b1622\" (UID: \"601a1248-dcd8-4100-bead-8888cb2b1622\") "
Oct 30 01:13:11.225547 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.225514    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/601a1248-dcd8-4100-bead-8888cb2b1622-openshift-service-ca" pod "601a1248-dcd8-4100-bead-8888cb2b1622" (UID: "601a1248-dcd8-4100-bead-8888cb2b1622"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:13:11.226044 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.226016    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/601a1248-dcd8-4100-bead-8888cb2b1622-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:11.228801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.227715    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-metrics-certs" pod "601a1248-dcd8-4100-bead-8888cb2b1622" (UID: "601a1248-dcd8-4100-bead-8888cb2b1622"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:11.228801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.227972    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/601a1248-dcd8-4100-bead-8888cb2b1622-kube-api-access-tdj6d" pod "601a1248-dcd8-4100-bead-8888cb2b1622" (UID: "601a1248-dcd8-4100-bead-8888cb2b1622"). InnerVolumeSpecName "kube-api-access-tdj6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:13:11.228801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.228068    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls" pod "601a1248-dcd8-4100-bead-8888cb2b1622" (UID: "601a1248-dcd8-4100-bead-8888cb2b1622"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:11.327688 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.327594    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdj6d\" (UniqueName: \"kubernetes.io/projected/601a1248-dcd8-4100-bead-8888cb2b1622-kube-api-access-tdj6d\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:11.327688 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.327630    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:11.327688 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.327643    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/601a1248-dcd8-4100-bead-8888cb2b1622-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:11.498767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.498729    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"]
Oct 30 01:13:11.511895 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:11.511855    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-b2bf8a-b98847dc7-zwtjx"]
Oct 30 01:13:12.022582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:12.022545    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="601a1248-dcd8-4100-bead-8888cb2b1622" path="/var/lib/kubelet/pods/601a1248-dcd8-4100-bead-8888cb2b1622/volumes"
Oct 30 01:13:14.542617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.542584    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"]
Oct 30 01:13:14.543409 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.543391    6011 cpu_manager.go:414] "containerMap: removing container" podUID="601a1248-dcd8-4100-bead-8888cb2b1622" containerName="maas-api"
Oct 30 01:13:14.543475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.543409    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="601a1248-dcd8-4100-bead-8888cb2b1622" containerName="maas-api"
Oct 30 01:13:14.543546 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.543535    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="601a1248-dcd8-4100-bead-8888cb2b1622" containerName="maas-api"
Oct 30 01:13:14.548743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.548709    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.563962 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.563929    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"]
Oct 30 01:13:14.574336 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.574300    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b312f7\"/\"kube-root-ca.crt\""
Oct 30 01:13:14.574336 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.574300    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b312f7\"/\"openshift-service-ca.crt\""
Oct 30 01:13:14.577978 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.577942    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b312f7\"/\"test-model-b312f7-kserve-self-signed-certs\""
Oct 30 01:13:14.578130 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.577985    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b312f7\"/\"default-dockercfg-h8xzr\""
Oct 30 01:13:14.673597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.673557    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2thnd\" (UniqueName: \"kubernetes.io/projected/5e648bdd-1a40-43f6-ab96-541d3d538966-kube-api-access-2thnd\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.673804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.673621    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e648bdd-1a40-43f6-ab96-541d3d538966-tls-certs\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.673804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.673781    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-tmp-dir\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.673985 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.673967    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-dshm\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.674050 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.673996    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-model-cache\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.674134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.674119    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-home\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.777022    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-dshm\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.777082    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-model-cache\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.777191    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-home\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.777346    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2thnd\" (UniqueName: \"kubernetes.io/projected/5e648bdd-1a40-43f6-ab96-541d3d538966-kube-api-access-2thnd\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.777404    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e648bdd-1a40-43f6-ab96-541d3d538966-tls-certs\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.777435    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-tmp-dir\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.778013    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-tmp-dir\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779879 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.779389    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-home\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.779879 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.779640    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-model-cache\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.784216 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.781274    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-dshm\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.787464 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.787394    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e648bdd-1a40-43f6-ab96-541d3d538966-tls-certs\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.802273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.802140    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2thnd\" (UniqueName: \"kubernetes.io/projected/5e648bdd-1a40-43f6-ab96-541d3d538966-kube-api-access-2thnd\") pod \"test-model-b312f7-kserve-974fd85-ntfhv\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") " pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:14.862945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:14.862910    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:15.166948 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:13:15.166902    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e648bdd_1a40_43f6_ab96_541d3d538966.slice/crio-b7890d0c53f6c126809076085287ba8b39f8cef02002b473cc50817486894ad5 WatchSource:0}: Error finding container b7890d0c53f6c126809076085287ba8b39f8cef02002b473cc50817486894ad5: Status 404 returned error can't find the container with id b7890d0c53f6c126809076085287ba8b39f8cef02002b473cc50817486894ad5
Oct 30 01:13:15.167163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:15.167132    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"]
Oct 30 01:13:15.181933 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:15.181902    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv" event={"ID":"5e648bdd-1a40-43f6-ab96-541d3d538966","Type":"ContainerStarted","Data":"b7890d0c53f6c126809076085287ba8b39f8cef02002b473cc50817486894ad5"}
Oct 30 01:13:16.190894 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:16.190853    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv" event={"ID":"5e648bdd-1a40-43f6-ab96-541d3d538966","Type":"ContainerStarted","Data":"f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb"}
Oct 30 01:13:16.233452 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:16.233391    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv" podStartSLOduration=2.233376907 podStartE2EDuration="2.233376907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:13:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:13:16.231277944 +0000 UTC m=+2038.334031916" watchObservedRunningTime="2026-10-30 01:13:16.233376907 +0000 UTC m=+2038.336130878"
Oct 30 01:13:16.858227 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:16.858168    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:16.863331 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:16.863303    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:16.871431 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:16.871414    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:17.209254 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:17.209156    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:13:18.130418 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:18.130385    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:13:24.863385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:24.863350    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:24.876447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:24.876420    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:25.287708 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:25.287682    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:13:27.911752 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:13:27.911713    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:13:49.846668 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:49.846628    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"]
Oct 30 01:13:49.847308 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:49.846943    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx" podUID="c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" containerName="maas-api" containerID="cri-o://12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3" gracePeriod=30
Oct 30 01:13:50.134434 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.134399    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:13:50.290698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.290665    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-kube-api-access-blzh7\" (UniqueName: \"kubernetes.io/projected/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-kube-api-access-blzh7\") pod \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") "
Oct 30 01:13:50.290916 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.290729    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-openshift-service-ca\") pod \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") "
Oct 30 01:13:50.290979 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.290949    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-metrics-certs\") pod \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") "
Oct 30 01:13:50.291048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.291027    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls\") pod \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\" (UID: \"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3\") "
Oct 30 01:13:50.291167 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.291130    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-openshift-service-ca" pod "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" (UID: "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:13:50.291597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.291570    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:50.293392 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.293359    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls" pod "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" (UID: "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:50.293842 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.293809    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-metrics-certs" pod "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" (UID: "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:50.293951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.293814    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-kube-api-access-blzh7" pod "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" (UID: "c8a84b94-b272-4927-b0cd-4c33ba3bb6d3"). InnerVolumeSpecName "kube-api-access-blzh7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:13:50.393914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.393770    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blzh7\" (UniqueName: \"kubernetes.io/projected/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-kube-api-access-blzh7\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:50.393914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.393813    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:50.393914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.393830    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:50.493845 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.493809    6011 generic.go:356] "Generic (PLEG): container finished" podID="c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" containerID="12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3" exitCode=0
Oct 30 01:13:50.494030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.493858    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"
Oct 30 01:13:50.494030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.493901    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx" event={"ID":"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3","Type":"ContainerDied","Data":"12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3"}
Oct 30 01:13:50.494030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.493964    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx" event={"ID":"c8a84b94-b272-4927-b0cd-4c33ba3bb6d3","Type":"ContainerDied","Data":"fd3eeb7db8f57b0602eaeb341e6f0aa6140e40f94dcddd2cdca6482e28f25842"}
Oct 30 01:13:50.494030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.493990    6011 scope.go:122] "RemoveContainer" containerID="12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3"
Oct 30 01:13:50.506379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.506350    6011 scope.go:122] "RemoveContainer" containerID="12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3"
Oct 30 01:13:50.506707 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:13:50.506687    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3\": container with ID starting with 12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3 not found: ID does not exist" containerID="12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3"
Oct 30 01:13:50.506775 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.506715    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3"} err="failed to get container status \"12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3\": rpc error: code = NotFound desc = could not find container \"12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3\": container with ID starting with 12f0c85bf1ec608d68d23f2fba05e07099de86970daff8b43ceaec7d90d13cb3 not found: ID does not exist"
Oct 30 01:13:50.549727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.549695    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"]
Oct 30 01:13:50.553496 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:50.553470    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-d54608-647c5cb9ff-46vcx"]
Oct 30 01:13:52.022749 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:52.022709    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" path="/var/lib/kubelet/pods/c8a84b94-b272-4927-b0cd-4c33ba3bb6d3/volumes"
Oct 30 01:13:52.442989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:52.442946    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"]
Oct 30 01:13:52.443246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:52.443216    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2" podUID="0028edc5-ace8-4e51-aaed-47079626d470" containerName="maas-api" containerID="cri-o://ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23" gracePeriod=30
Oct 30 01:13:52.801297 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:13:52.801248    6011 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0028edc5_ace8_4e51_aaed_47079626d470.slice/crio-conmon-ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0028edc5_ace8_4e51_aaed_47079626d470.slice/crio-ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23.scope\": RecentStats: unable to find data in memory cache]"
Oct 30 01:13:53.016699 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.016671    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:13:53.061701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.061662    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0028edc5-ace8-4e51-aaed-47079626d470-kube-api-access-nftsh\" (UniqueName: \"kubernetes.io/projected/0028edc5-ace8-4e51-aaed-47079626d470-kube-api-access-nftsh\") pod \"0028edc5-ace8-4e51-aaed-47079626d470\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") "
Oct 30 01:13:53.062414 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.061748    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0028edc5-ace8-4e51-aaed-47079626d470-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028edc5-ace8-4e51-aaed-47079626d470-openshift-service-ca\") pod \"0028edc5-ace8-4e51-aaed-47079626d470\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") "
Oct 30 01:13:53.062414 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.061841    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-maas-api-tls\") pod \"0028edc5-ace8-4e51-aaed-47079626d470\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") "
Oct 30 01:13:53.062550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.062407    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0028edc5-ace8-4e51-aaed-47079626d470-openshift-service-ca" pod "0028edc5-ace8-4e51-aaed-47079626d470" (UID: "0028edc5-ace8-4e51-aaed-47079626d470"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:13:53.062999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.062975    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0028edc5-ace8-4e51-aaed-47079626d470-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:53.064513 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.064475    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0028edc5-ace8-4e51-aaed-47079626d470-kube-api-access-nftsh" pod "0028edc5-ace8-4e51-aaed-47079626d470" (UID: "0028edc5-ace8-4e51-aaed-47079626d470"). InnerVolumeSpecName "kube-api-access-nftsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:13:53.064883 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.064786    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-maas-api-tls" pod "0028edc5-ace8-4e51-aaed-47079626d470" (UID: "0028edc5-ace8-4e51-aaed-47079626d470"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:53.164422 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.164378    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-metrics-certs\") pod \"0028edc5-ace8-4e51-aaed-47079626d470\" (UID: \"0028edc5-ace8-4e51-aaed-47079626d470\") "
Oct 30 01:13:53.164837 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.164814    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nftsh\" (UniqueName: \"kubernetes.io/projected/0028edc5-ace8-4e51-aaed-47079626d470-kube-api-access-nftsh\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:53.164922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.164844    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:53.167503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.166673    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-metrics-certs" pod "0028edc5-ace8-4e51-aaed-47079626d470" (UID: "0028edc5-ace8-4e51-aaed-47079626d470"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:13:53.265794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.265710    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0028edc5-ace8-4e51-aaed-47079626d470-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:13:53.520530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.520435    6011 generic.go:356] "Generic (PLEG): container finished" podID="0028edc5-ace8-4e51-aaed-47079626d470" containerID="ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23" exitCode=0
Oct 30 01:13:53.520530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.520490    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"
Oct 30 01:13:53.520530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.520517    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2" event={"ID":"0028edc5-ace8-4e51-aaed-47079626d470","Type":"ContainerDied","Data":"ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23"}
Oct 30 01:13:53.520885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.520596    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2" event={"ID":"0028edc5-ace8-4e51-aaed-47079626d470","Type":"ContainerDied","Data":"7d1fd1da0dbff2c4f14ebeecdef124df413bc939f56086c0badbd11af7eb7c07"}
Oct 30 01:13:53.520885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.520619    6011 scope.go:122] "RemoveContainer" containerID="ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23"
Oct 30 01:13:53.532383 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.532363    6011 scope.go:122] "RemoveContainer" containerID="ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23"
Oct 30 01:13:53.532677 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:13:53.532658    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23\": container with ID starting with ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23 not found: ID does not exist" containerID="ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23"
Oct 30 01:13:53.532723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.532685    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23"} err="failed to get container status \"ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23\": rpc error: code = NotFound desc = could not find container \"ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23\": container with ID starting with ed316491331a15bb46171edae24177b546d5f05978c4acac13efce8e65401d23 not found: ID does not exist"
Oct 30 01:13:53.576937 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.576896    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"]
Oct 30 01:13:53.591763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:53.591730    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-286d36-9c6fb6d56-z55q2"]
Oct 30 01:13:54.022646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:13:54.022612    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0028edc5-ace8-4e51-aaed-47079626d470" path="/var/lib/kubelet/pods/0028edc5-ace8-4e51-aaed-47079626d470/volumes"
Oct 30 01:13:57.912368 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:13:57.912325    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:14:27.911837 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:14:27.911791    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:14:49.207040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.206993    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"]
Oct 30 01:14:49.207677 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.207662    6011 cpu_manager.go:414] "containerMap: removing container" podUID="0028edc5-ace8-4e51-aaed-47079626d470" containerName="maas-api"
Oct 30 01:14:49.207729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.207678    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0028edc5-ace8-4e51-aaed-47079626d470" containerName="maas-api"
Oct 30 01:14:49.207729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.207706    6011 cpu_manager.go:414] "containerMap: removing container" podUID="c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" containerName="maas-api"
Oct 30 01:14:49.207729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.207711    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" containerName="maas-api"
Oct 30 01:14:49.207821 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.207812    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="0028edc5-ace8-4e51-aaed-47079626d470" containerName="maas-api"
Oct 30 01:14:49.207854 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.207824    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8a84b94-b272-4927-b0cd-4c33ba3bb6d3" containerName="maas-api"
Oct 30 01:14:49.211031 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.211011    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.229416 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.229384    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"]
Oct 30 01:14:49.239434 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.239398    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w5-bdf6f3\"/\"kube-root-ca.crt\""
Oct 30 01:14:49.239674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.239408    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w5-bdf6f3\"/\"openshift-service-ca.crt\""
Oct 30 01:14:49.239674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.239424    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w5-bdf6f3\"/\"default-dockercfg-lslq2\""
Oct 30 01:14:49.240444 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.240421    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w5-bdf6f3\"/\"auth-test-model-bdf6f3-kserve-self-signed-certs\""
Oct 30 01:14:49.392922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.392886    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-model-cache\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.392922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.392928    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tmp-dir\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.393148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.392967    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqvvr\" (UniqueName: \"kubernetes.io/projected/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-kube-api-access-wqvvr\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.393148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.393130    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tls-certs\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.393321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.393303    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-home\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.393365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.393337    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-dshm\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.494436 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494348    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tls-certs\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.494624 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494496    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-home\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.494624 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494526    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-dshm\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.494755 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494643    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-model-cache\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.494755 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494670    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tmp-dir\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.494755 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494710    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqvvr\" (UniqueName: \"kubernetes.io/projected/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-kube-api-access-wqvvr\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.494994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494968    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-home\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.495079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.494978    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-model-cache\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.495079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.495057    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tmp-dir\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.497119 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.497093    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-dshm\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.497264 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.497167    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tls-certs\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.518057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.517975    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqvvr\" (UniqueName: \"kubernetes.io/projected/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-kube-api-access-wqvvr\") pod \"auth-test-model-bdf6f3-kserve-854c97475c-hfpmr\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") " pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.533460 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.533428    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:49.898232 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.898192    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"]
Oct 30 01:14:49.972861 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:49.971261    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr" event={"ID":"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9","Type":"ContainerStarted","Data":"85d052e805795cd4c44e837bc62c8f60ef47a9f0ac14f966b4b99f2fa6f96fcd"}
Oct 30 01:14:50.984391 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:50.984343    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr" event={"ID":"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9","Type":"ContainerStarted","Data":"a23752131c3b7a5e4545b5557ff6ffba9f3f33631b5100a0aded9d764cd925c9"}
Oct 30 01:14:51.061663 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:51.061586    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr" podStartSLOduration=2.061557705 podStartE2EDuration="2.061557705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:14:51.030658752 +0000 UTC m=+2133.133412744" watchObservedRunningTime="2026-10-30 01:14:51.061557705 +0000 UTC m=+2133.164311677"
Oct 30 01:14:51.539041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:51.538997    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:57.913481 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:14:57.913435    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:14:59.538900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.538862    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:59.551895 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.551858    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:14:59.831594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.831496    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"]
Oct 30 01:14:59.836109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.836086    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:14:59.851993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.851972    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"]
Oct 30 01:14:59.862072 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.862044    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-cfe8f0-serving-cert\""
Oct 30 01:14:59.947315 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.947278    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:14:59.947492 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.947415    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:14:59.947492 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.947465    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34b3c21-b0d9-4492-9c42-81ed216616d6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:14:59.947567 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:14:59.947552    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r68k6\" (UniqueName: \"kubernetes.io/projected/b34b3c21-b0d9-4492-9c42-81ed216616d6-kube-api-access-r68k6\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.051694 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.051653    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.055988 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.051994    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34b3c21-b0d9-4492-9c42-81ed216616d6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.055988 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.053907    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r68k6\" (UniqueName: \"kubernetes.io/projected/b34b3c21-b0d9-4492-9c42-81ed216616d6-kube-api-access-r68k6\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.055988 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.054224    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.057478 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.057449    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34b3c21-b0d9-4492-9c42-81ed216616d6-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.057478 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.057466    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.057671 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.057527    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.073992 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.073961    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:15:00.075595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.075557    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r68k6\" (UniqueName: \"kubernetes.io/projected/b34b3c21-b0d9-4492-9c42-81ed216616d6-kube-api-access-r68k6\") pod \"maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.153726 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.153633    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:00.490941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:00.490909    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"]
Oct 30 01:15:00.491301 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:15:00.491276    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34b3c21_b0d9_4492_9c42_81ed216616d6.slice/crio-0773ac6c247fa36421bc613c03a65df3eac72e81baeb0b0ac4d88126227b5625 WatchSource:0}: Error finding container 0773ac6c247fa36421bc613c03a65df3eac72e81baeb0b0ac4d88126227b5625: Status 404 returned error can't find the container with id 0773ac6c247fa36421bc613c03a65df3eac72e81baeb0b0ac4d88126227b5625
Oct 30 01:15:01.065769 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:01.065664    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x" event={"ID":"b34b3c21-b0d9-4492-9c42-81ed216616d6","Type":"ContainerStarted","Data":"c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b"}
Oct 30 01:15:01.065769 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:01.065726    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x" event={"ID":"b34b3c21-b0d9-4492-9c42-81ed216616d6","Type":"ContainerStarted","Data":"0773ac6c247fa36421bc613c03a65df3eac72e81baeb0b0ac4d88126227b5625"}
Oct 30 01:15:01.066302 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:01.065937    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:01.125509 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:01.125440    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x" podStartSLOduration=1.822411608 podStartE2EDuration="2.125417905s" totalImagesPullingTime="303.006297ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:14:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:01.107322512 +0000 UTC m=+2143.210076519" watchObservedRunningTime="2026-10-30 01:15:01.125417905 +0000 UTC m=+2143.228171878"
Oct 30 01:15:07.083075 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:07.083043    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:15:17.841675 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.841640    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"]
Oct 30 01:15:17.846156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.846135    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:17.863604 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.863577    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"]
Oct 30 01:15:17.872563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.872522    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-cb31e3-serving-cert\""
Oct 30 01:15:17.900555 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.900514    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzkkz\" (UniqueName: \"kubernetes.io/projected/d3f8731d-4f88-41c3-93a6-66c68bc3f428-kube-api-access-hzkkz\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:17.900761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.900657    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:17.900761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.900735    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f8731d-4f88-41c3-93a6-66c68bc3f428-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:17.900883 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:17.900867    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.002804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.002773    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.002995 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.002942    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzkkz\" (UniqueName: \"kubernetes.io/projected/d3f8731d-4f88-41c3-93a6-66c68bc3f428-kube-api-access-hzkkz\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.003042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.003001    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.003093 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.003078    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f8731d-4f88-41c3-93a6-66c68bc3f428-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.003799 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.003774    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f8731d-4f88-41c3-93a6-66c68bc3f428-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.005550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.005517    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.005550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.005564    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.023915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.023884    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzkkz\" (UniqueName: \"kubernetes.io/projected/d3f8731d-4f88-41c3-93a6-66c68bc3f428-kube-api-access-hzkkz\") pod \"maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.163510 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.163421    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:18.472855 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:15:18.472819    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3f8731d_4f88_41c3_93a6_66c68bc3f428.slice/crio-14c44892c9b5293a027620afa0b6c9a248e3cf3f6d3df2cee72acedfa6fba2bb WatchSource:0}: Error finding container 14c44892c9b5293a027620afa0b6c9a248e3cf3f6d3df2cee72acedfa6fba2bb: Status 404 returned error can't find the container with id 14c44892c9b5293a027620afa0b6c9a248e3cf3f6d3df2cee72acedfa6fba2bb
Oct 30 01:15:18.473150 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:18.473129    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"]
Oct 30 01:15:19.229079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:19.228978    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc" event={"ID":"d3f8731d-4f88-41c3-93a6-66c68bc3f428","Type":"ContainerStarted","Data":"876b76a523533bd0adbd2d14c1bca7a3c882eb240c5025d202108d13679a7743"}
Oct 30 01:15:19.229079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:19.229061    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc" event={"ID":"d3f8731d-4f88-41c3-93a6-66c68bc3f428","Type":"ContainerStarted","Data":"14c44892c9b5293a027620afa0b6c9a248e3cf3f6d3df2cee72acedfa6fba2bb"}
Oct 30 01:15:19.230116 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:19.230087    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:19.277724 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:19.277669    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc" podStartSLOduration=1.800670279 podStartE2EDuration="2.277654185s" totalImagesPullingTime="476.983906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:15:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:15:19.27441788 +0000 UTC m=+2161.377171851" watchObservedRunningTime="2026-10-30 01:15:19.277654185 +0000 UTC m=+2161.380408157"
Oct 30 01:15:21.697873 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:21.697838    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"]
Oct 30 01:15:26.250104 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:26.250072    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:15:27.914588 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:15:27.914498    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:15:36.720946 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:36.720893    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr" podUID="2615321f-445d-4aa1-85f7-cb1a6fb94207" containerName="main" containerID="cri-o://20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704" gracePeriod=45
Oct 30 01:15:36.766603 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:36.766565    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr" podUID="2615321f-445d-4aa1-85f7-cb1a6fb94207" containerName="main" probeResult="failure" output="Get \"https://10.134.0.122:8000/ready\": dial tcp 10.134.0.122:8000: connect: connection refused"
Oct 30 01:15:37.031331 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.031300    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:15:37.106622 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.106584    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-dshm\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-dshm\") pod \"2615321f-445d-4aa1-85f7-cb1a6fb94207\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") "
Oct 30 01:15:37.106830 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.106738    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-home\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-home\") pod \"2615321f-445d-4aa1-85f7-cb1a6fb94207\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") "
Oct 30 01:15:37.106830 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.106797    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-model-cache\") pod \"2615321f-445d-4aa1-85f7-cb1a6fb94207\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") "
Oct 30 01:15:37.106948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.106836    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2615321f-445d-4aa1-85f7-cb1a6fb94207-tls-certs\" (UniqueName: \"kubernetes.io/secret/2615321f-445d-4aa1-85f7-cb1a6fb94207-tls-certs\") pod \"2615321f-445d-4aa1-85f7-cb1a6fb94207\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") "
Oct 30 01:15:37.106948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.106920    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-tmp-dir\") pod \"2615321f-445d-4aa1-85f7-cb1a6fb94207\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") "
Oct 30 01:15:37.107064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.106993    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2615321f-445d-4aa1-85f7-cb1a6fb94207-kube-api-access-fv6gs\" (UniqueName: \"kubernetes.io/projected/2615321f-445d-4aa1-85f7-cb1a6fb94207-kube-api-access-fv6gs\") pod \"2615321f-445d-4aa1-85f7-cb1a6fb94207\" (UID: \"2615321f-445d-4aa1-85f7-cb1a6fb94207\") "
Oct 30 01:15:37.107422 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.107389    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-model-cache" pod "2615321f-445d-4aa1-85f7-cb1a6fb94207" (UID: "2615321f-445d-4aa1-85f7-cb1a6fb94207"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:37.107895 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.107864    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-home" pod "2615321f-445d-4aa1-85f7-cb1a6fb94207" (UID: "2615321f-445d-4aa1-85f7-cb1a6fb94207"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:37.108647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.108624    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:15:37.108814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.108655    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:15:37.108974 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.108935    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-tmp-dir" pod "2615321f-445d-4aa1-85f7-cb1a6fb94207" (UID: "2615321f-445d-4aa1-85f7-cb1a6fb94207"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:37.111159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.111116    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2615321f-445d-4aa1-85f7-cb1a6fb94207-tls-certs" pod "2615321f-445d-4aa1-85f7-cb1a6fb94207" (UID: "2615321f-445d-4aa1-85f7-cb1a6fb94207"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:15:37.118386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.118357    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-dshm" pod "2615321f-445d-4aa1-85f7-cb1a6fb94207" (UID: "2615321f-445d-4aa1-85f7-cb1a6fb94207"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:15:37.132127 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.132102    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2615321f-445d-4aa1-85f7-cb1a6fb94207-kube-api-access-fv6gs" pod "2615321f-445d-4aa1-85f7-cb1a6fb94207" (UID: "2615321f-445d-4aa1-85f7-cb1a6fb94207"). InnerVolumeSpecName "kube-api-access-fv6gs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:15:37.210091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.210056    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:15:37.210091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.210085    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv6gs\" (UniqueName: \"kubernetes.io/projected/2615321f-445d-4aa1-85f7-cb1a6fb94207-kube-api-access-fv6gs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:15:37.210091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.210096    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2615321f-445d-4aa1-85f7-cb1a6fb94207-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:15:37.210380 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.210105    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2615321f-445d-4aa1-85f7-cb1a6fb94207-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:15:37.367379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.367331    6011 generic.go:356] "Generic (PLEG): container finished" podID="2615321f-445d-4aa1-85f7-cb1a6fb94207" containerID="20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704" exitCode=0
Oct 30 01:15:37.367575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.367465    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"
Oct 30 01:15:37.367575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.367474    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr" event={"ID":"2615321f-445d-4aa1-85f7-cb1a6fb94207","Type":"ContainerDied","Data":"20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704"}
Oct 30 01:15:37.367575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.367525    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr" event={"ID":"2615321f-445d-4aa1-85f7-cb1a6fb94207","Type":"ContainerDied","Data":"7c14dc3ce4f9729f491680b79032395960ed3741be67d58434ffdf48e8b8058f"}
Oct 30 01:15:37.367575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.367546    6011 scope.go:122] "RemoveContainer" containerID="20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704"
Oct 30 01:15:37.379120 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.379098    6011 scope.go:122] "RemoveContainer" containerID="20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704"
Oct 30 01:15:37.379475 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:15:37.379454    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704\": container with ID starting with 20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704 not found: ID does not exist" containerID="20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704"
Oct 30 01:15:37.379528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.379487    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704"} err="failed to get container status \"20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704\": rpc error: code = NotFound desc = could not find container \"20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704\": container with ID starting with 20302350491b58effb599e52384690cb13e6f38cad777fdf25b557793aaea704 not found: ID does not exist"
Oct 30 01:15:37.423356 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.423326    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"]
Oct 30 01:15:37.436399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:37.436371    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/auth-test-model-283eb6-kserve-6bf9655cbf-cvjbr"]
Oct 30 01:15:38.022215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:15:38.022159    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2615321f-445d-4aa1-85f7-cb1a6fb94207" path="/var/lib/kubelet/pods/2615321f-445d-4aa1-85f7-cb1a6fb94207/volumes"
Oct 30 01:15:57.912843 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:15:57.912799    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:16:07.728548 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:07.726950    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"]
Oct 30 01:16:13.303788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:13.303752    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"]
Oct 30 01:16:14.055975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.055937    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"]
Oct 30 01:16:14.056634 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.056619    6011 cpu_manager.go:414] "containerMap: removing container" podUID="2615321f-445d-4aa1-85f7-cb1a6fb94207" containerName="main"
Oct 30 01:16:14.056693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.056636    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2615321f-445d-4aa1-85f7-cb1a6fb94207" containerName="main"
Oct 30 01:16:14.056768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.056758    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="2615321f-445d-4aa1-85f7-cb1a6fb94207" containerName="main"
Oct 30 01:16:14.060729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.060694    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.100675 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.100643    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-ca324d-openshift-default-dockercfg-w2cst\""
Oct 30 01:16:14.104787 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.104756    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"]
Oct 30 01:16:14.131010 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.130978    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-credential-socket\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131029    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-envoy\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131100    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-certs\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131312 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131267    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-data\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131348 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131311    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfch9\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-kube-api-access-qfch9\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131348 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131339    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-token\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131450 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131413    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-socket\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131593 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131574    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-podinfo\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.131755 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.131739    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istiod-ca-cert\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233047 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233016    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-data\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233058    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfch9\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-kube-api-access-qfch9\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233077    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-token\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233111    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-socket\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233191    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-podinfo\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233320    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istiod-ca-cert\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233371    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-credential-socket\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233409    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-envoy\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233437    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-certs\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.233528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233494    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-data\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.234035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233535    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-socket\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.234035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233739    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-certs\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.234035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.233809    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-credential-socket\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.234307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.234284    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istiod-ca-cert\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.235783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.235755    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-envoy\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.235970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.235950    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-podinfo\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.254166 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.254134    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-token\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.254293 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.254277    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfch9\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-kube-api-access-qfch9\") pod \"e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") " pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.375959 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.375880    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:14.728516 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:16:14.728485    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1aae6be_9828_4b54_8c2e_039c6bb9dfbc.slice/crio-fb59fbb678710a2915363d067dd8fef2563b94b4018a1c1262d5f144f7dabe00 WatchSource:0}: Error finding container fb59fbb678710a2915363d067dd8fef2563b94b4018a1c1262d5f144f7dabe00: Status 404 returned error can't find the container with id fb59fbb678710a2915363d067dd8fef2563b94b4018a1c1262d5f144f7dabe00
Oct 30 01:16:14.728804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.728781    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"]
Oct 30 01:16:14.731259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.731227    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:16:14.731323 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.731293    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:16:14.731364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:14.731321    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:16:15.685909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:15.685868    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" event={"ID":"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc","Type":"ContainerStarted","Data":"6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7"}
Oct 30 01:16:15.686379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:15.685927    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" event={"ID":"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc","Type":"ContainerStarted","Data":"fb59fbb678710a2915363d067dd8fef2563b94b4018a1c1262d5f144f7dabe00"}
Oct 30 01:16:15.776449 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:15.776387    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" podStartSLOduration=1.7763728140000001 podStartE2EDuration="1.776372814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:16:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:16:15.760708707 +0000 UTC m=+2217.863462680" watchObservedRunningTime="2026-10-30 01:16:15.776372814 +0000 UTC m=+2217.879126785"
Oct 30 01:16:16.376359 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:16.376318    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:16.378262 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:16.378234    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 01:16:16.378402 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:16.378285    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 01:16:17.376850 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:17.376805    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 01:16:17.377284 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:17.376880    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 01:16:18.376735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:18.376696    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 01:16:18.376931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:18.376757    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 01:16:19.376724 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:19.376682    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 01:16:19.376929 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:19.376753    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 01:16:20.377044 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:20.377000    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused" start-of-body=
Oct 30 01:16:20.377530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:20.377068    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.129:15021/healthz/ready\": dial tcp 10.134.0.129:15021: connect: connection refused"
Oct 30 01:16:21.506123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:21.506090    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:21.507080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:21.507058    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:21.511970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:21.511942    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:16:22.753088 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:22.753041    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29" podUID="14f9d6a2-1416-4314-9882-94e33cb9ef8b" containerName="main" containerID="cri-o://706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee" gracePeriod=45
Oct 30 01:16:23.028786 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.028752    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:16:23.100755 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.100725    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tmp-dir\") pod \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") "
Oct 30 01:16:23.100941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.100778    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-model-cache\") pod \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") "
Oct 30 01:16:23.100941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.100906    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-dshm\") pod \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") "
Oct 30 01:16:23.100941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.100933    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-home\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-home\") pod \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") "
Oct 30 01:16:23.101109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.100976    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14f9d6a2-1416-4314-9882-94e33cb9ef8b-kube-api-access-rgrxh\" (UniqueName: \"kubernetes.io/projected/14f9d6a2-1416-4314-9882-94e33cb9ef8b-kube-api-access-rgrxh\") pod \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") "
Oct 30 01:16:23.101109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.101004    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tmp-dir" pod "14f9d6a2-1416-4314-9882-94e33cb9ef8b" (UID: "14f9d6a2-1416-4314-9882-94e33cb9ef8b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:23.101109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.101085    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tls-certs\" (UniqueName: \"kubernetes.io/secret/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tls-certs\") pod \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\" (UID: \"14f9d6a2-1416-4314-9882-94e33cb9ef8b\") "
Oct 30 01:16:23.101342 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.101302    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-home" pod "14f9d6a2-1416-4314-9882-94e33cb9ef8b" (UID: "14f9d6a2-1416-4314-9882-94e33cb9ef8b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:23.101797 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.101746    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:23.101797 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.101773    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:23.102112 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.102068    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-model-cache" pod "14f9d6a2-1416-4314-9882-94e33cb9ef8b" (UID: "14f9d6a2-1416-4314-9882-94e33cb9ef8b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:23.104052 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.104028    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-dshm" pod "14f9d6a2-1416-4314-9882-94e33cb9ef8b" (UID: "14f9d6a2-1416-4314-9882-94e33cb9ef8b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:23.104390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.104369    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14f9d6a2-1416-4314-9882-94e33cb9ef8b-kube-api-access-rgrxh" pod "14f9d6a2-1416-4314-9882-94e33cb9ef8b" (UID: "14f9d6a2-1416-4314-9882-94e33cb9ef8b"). InnerVolumeSpecName "kube-api-access-rgrxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:16:23.104626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.104608    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tls-certs" pod "14f9d6a2-1416-4314-9882-94e33cb9ef8b" (UID: "14f9d6a2-1416-4314-9882-94e33cb9ef8b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:16:23.203298 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.203260    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/14f9d6a2-1416-4314-9882-94e33cb9ef8b-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:23.203298 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.203293    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:23.203298 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.203301    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/14f9d6a2-1416-4314-9882-94e33cb9ef8b-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:23.203543 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.203310    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgrxh\" (UniqueName: \"kubernetes.io/projected/14f9d6a2-1416-4314-9882-94e33cb9ef8b-kube-api-access-rgrxh\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:23.747003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.746968    6011 generic.go:356] "Generic (PLEG): container finished" podID="14f9d6a2-1416-4314-9882-94e33cb9ef8b" containerID="706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee" exitCode=0
Oct 30 01:16:23.747242 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.747012    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"
Oct 30 01:16:23.747242 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.747060    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29" event={"ID":"14f9d6a2-1416-4314-9882-94e33cb9ef8b","Type":"ContainerDied","Data":"706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee"}
Oct 30 01:16:23.747242 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.747101    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29" event={"ID":"14f9d6a2-1416-4314-9882-94e33cb9ef8b","Type":"ContainerDied","Data":"1eba9f3964f602afacc4c9a403f6ea84100c1e46326934e75a7ae9482f5ca3d4"}
Oct 30 01:16:23.747242 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.747130    6011 scope.go:122] "RemoveContainer" containerID="706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee"
Oct 30 01:16:23.758822 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.758713    6011 scope.go:122] "RemoveContainer" containerID="706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee"
Oct 30 01:16:23.759058 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:16:23.759034    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee\": container with ID starting with 706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee not found: ID does not exist" containerID="706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee"
Oct 30 01:16:23.759098 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.759066    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee"} err="failed to get container status \"706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee\": rpc error: code = NotFound desc = could not find container \"706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee\": container with ID starting with 706f1c87f310bff760b5ffe97c0dd832c265e1c2cff17ca5199b30d3525117ee not found: ID does not exist"
Oct 30 01:16:23.806565 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.806525    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"]
Oct 30 01:16:23.821283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:23.821241    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w0-43d152/auth-test-model-43d152-kserve-6454d9fd49-qnj29"]
Oct 30 01:16:24.031451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:24.031367    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14f9d6a2-1416-4314-9882-94e33cb9ef8b" path="/var/lib/kubelet/pods/14f9d6a2-1416-4314-9882-94e33cb9ef8b/volumes"
Oct 30 01:16:27.914962 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:16:27.914922    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:16:28.329818 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.329767    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv" podUID="5e648bdd-1a40-43f6-ab96-541d3d538966" containerName="main" containerID="cri-o://f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb" gracePeriod=45
Oct 30 01:16:28.631670 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.631636    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:16:28.685359 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.685327    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-dshm\") pod \"5e648bdd-1a40-43f6-ab96-541d3d538966\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") "
Oct 30 01:16:28.685549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.685374    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5e648bdd-1a40-43f6-ab96-541d3d538966-tls-certs\" (UniqueName: \"kubernetes.io/secret/5e648bdd-1a40-43f6-ab96-541d3d538966-tls-certs\") pod \"5e648bdd-1a40-43f6-ab96-541d3d538966\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") "
Oct 30 01:16:28.685549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.685501    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5e648bdd-1a40-43f6-ab96-541d3d538966-kube-api-access-2thnd\" (UniqueName: \"kubernetes.io/projected/5e648bdd-1a40-43f6-ab96-541d3d538966-kube-api-access-2thnd\") pod \"5e648bdd-1a40-43f6-ab96-541d3d538966\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") "
Oct 30 01:16:28.685674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.685631    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-model-cache\") pod \"5e648bdd-1a40-43f6-ab96-541d3d538966\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") "
Oct 30 01:16:28.685674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.685665    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-tmp-dir\") pod \"5e648bdd-1a40-43f6-ab96-541d3d538966\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") "
Oct 30 01:16:28.685780 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.685727    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-home\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-home\") pod \"5e648bdd-1a40-43f6-ab96-541d3d538966\" (UID: \"5e648bdd-1a40-43f6-ab96-541d3d538966\") "
Oct 30 01:16:28.686342 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.686307    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-tmp-dir" pod "5e648bdd-1a40-43f6-ab96-541d3d538966" (UID: "5e648bdd-1a40-43f6-ab96-541d3d538966"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:28.686483 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.686308    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-model-cache" pod "5e648bdd-1a40-43f6-ab96-541d3d538966" (UID: "5e648bdd-1a40-43f6-ab96-541d3d538966"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:28.686483 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.686373    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-home" pod "5e648bdd-1a40-43f6-ab96-541d3d538966" (UID: "5e648bdd-1a40-43f6-ab96-541d3d538966"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:28.689161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.689130    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e648bdd-1a40-43f6-ab96-541d3d538966-tls-certs" pod "5e648bdd-1a40-43f6-ab96-541d3d538966" (UID: "5e648bdd-1a40-43f6-ab96-541d3d538966"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:16:28.691287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.691268    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-dshm" pod "5e648bdd-1a40-43f6-ab96-541d3d538966" (UID: "5e648bdd-1a40-43f6-ab96-541d3d538966"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:16:28.691364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.691351    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e648bdd-1a40-43f6-ab96-541d3d538966-kube-api-access-2thnd" pod "5e648bdd-1a40-43f6-ab96-541d3d538966" (UID: "5e648bdd-1a40-43f6-ab96-541d3d538966"). InnerVolumeSpecName "kube-api-access-2thnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:16:28.787632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.787593    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:28.787632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.787631    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:28.787632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.787641    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:28.787966 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.787652    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e648bdd-1a40-43f6-ab96-541d3d538966-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:28.787966 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.787662    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e648bdd-1a40-43f6-ab96-541d3d538966-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:28.787966 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.787673    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2thnd\" (UniqueName: \"kubernetes.io/projected/5e648bdd-1a40-43f6-ab96-541d3d538966-kube-api-access-2thnd\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:16:28.822692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.822658    6011 generic.go:356] "Generic (PLEG): container finished" podID="5e648bdd-1a40-43f6-ab96-541d3d538966" containerID="f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb" exitCode=0
Oct 30 01:16:28.822912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.822706    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"
Oct 30 01:16:28.822912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.822748    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv" event={"ID":"5e648bdd-1a40-43f6-ab96-541d3d538966","Type":"ContainerDied","Data":"f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb"}
Oct 30 01:16:28.822912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.822797    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv" event={"ID":"5e648bdd-1a40-43f6-ab96-541d3d538966","Type":"ContainerDied","Data":"b7890d0c53f6c126809076085287ba8b39f8cef02002b473cc50817486894ad5"}
Oct 30 01:16:28.822912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.822818    6011 scope.go:122] "RemoveContainer" containerID="f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb"
Oct 30 01:16:28.835837 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.835810    6011 scope.go:122] "RemoveContainer" containerID="f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb"
Oct 30 01:16:28.836152 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:16:28.836132    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb\": container with ID starting with f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb not found: ID does not exist" containerID="f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb"
Oct 30 01:16:28.836223 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.836162    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb"} err="failed to get container status \"f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb\": rpc error: code = NotFound desc = could not find container \"f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb\": container with ID starting with f40b9a8a3a946a015276f57f818601b49808b94bda84c09016d91fea0c18e3fb not found: ID does not exist"
Oct 30 01:16:28.881449 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.881363    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"]
Oct 30 01:16:28.886762 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:28.886726    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b312f7/test-model-b312f7-kserve-974fd85-ntfhv"]
Oct 30 01:16:30.034001 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:30.033961    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5e648bdd-1a40-43f6-ab96-541d3d538966" path="/var/lib/kubelet/pods/5e648bdd-1a40-43f6-ab96-541d3d538966/volumes"
Oct 30 01:16:39.039053 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.039013    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"]
Oct 30 01:16:39.040091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.040061    6011 cpu_manager.go:414] "containerMap: removing container" podUID="5e648bdd-1a40-43f6-ab96-541d3d538966" containerName="main"
Oct 30 01:16:39.040091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.040091    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5e648bdd-1a40-43f6-ab96-541d3d538966" containerName="main"
Oct 30 01:16:39.040317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.040169    6011 cpu_manager.go:414] "containerMap: removing container" podUID="14f9d6a2-1416-4314-9882-94e33cb9ef8b" containerName="main"
Oct 30 01:16:39.040317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.040194    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14f9d6a2-1416-4314-9882-94e33cb9ef8b" containerName="main"
Oct 30 01:16:39.040421 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.040376    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="14f9d6a2-1416-4314-9882-94e33cb9ef8b" containerName="main"
Oct 30 01:16:39.040421 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.040397    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="5e648bdd-1a40-43f6-ab96-541d3d538966" containerName="main"
Oct 30 01:16:39.046208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.046163    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.066218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.066165    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"]
Oct 30 01:16:39.074585 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.074528    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-cfe8f0\"/\"default-dockercfg-mfnh4\""
Oct 30 01:16:39.074585 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.074585    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-cfe8f0\"/\"auth-test-model-cfe8f0-kserve-self-signed-certs\""
Oct 30 01:16:39.075314 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.075286    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-cfe8f0\"/\"kube-root-ca.crt\""
Oct 30 01:16:39.075314 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.075314    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-cfe8f0\"/\"openshift-service-ca.crt\""
Oct 30 01:16:39.135471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.135437    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54cms\" (UniqueName: \"kubernetes.io/projected/9b417080-16fe-4b23-b53a-4a06cc221325-kube-api-access-54cms\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.135681 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.135500    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-tmp-dir\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.135681 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.135635    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b417080-16fe-4b23-b53a-4a06cc221325-tls-certs\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.135783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.135686    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-home\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.135859 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.135839    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-model-cache\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.135980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.135958    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-dshm\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.237582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.237536    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-home\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.237792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.237640    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-model-cache\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.237792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.237724    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-dshm\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.237881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.237830    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54cms\" (UniqueName: \"kubernetes.io/projected/9b417080-16fe-4b23-b53a-4a06cc221325-kube-api-access-54cms\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.237929 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.237902    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-tmp-dir\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.238002 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.237986    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b417080-16fe-4b23-b53a-4a06cc221325-tls-certs\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.238077 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.238041    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-home\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.238317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.238284    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-model-cache\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.238317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.238303    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-tmp-dir\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.240275 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.240249    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-dshm\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.240923 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.240898    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b417080-16fe-4b23-b53a-4a06cc221325-tls-certs\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.260056 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.260012    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54cms\" (UniqueName: \"kubernetes.io/projected/9b417080-16fe-4b23-b53a-4a06cc221325-kube-api-access-54cms\") pod \"auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") " pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.371983 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.371890    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:39.692351 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:16:39.692306    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9b417080_16fe_4b23_b53a_4a06cc221325.slice/crio-69d59e29f8f0c574b02c96e5b5611c710542fd3d967a471ba3e3dde4b376e1b5 WatchSource:0}: Error finding container 69d59e29f8f0c574b02c96e5b5611c710542fd3d967a471ba3e3dde4b376e1b5: Status 404 returned error can't find the container with id 69d59e29f8f0c574b02c96e5b5611c710542fd3d967a471ba3e3dde4b376e1b5
Oct 30 01:16:39.693251 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.693226    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"]
Oct 30 01:16:39.908311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.908259    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd" event={"ID":"9b417080-16fe-4b23-b53a-4a06cc221325","Type":"ContainerStarted","Data":"a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d"}
Oct 30 01:16:39.908311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.908313    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd" event={"ID":"9b417080-16fe-4b23-b53a-4a06cc221325","Type":"ContainerStarted","Data":"69d59e29f8f0c574b02c96e5b5611c710542fd3d967a471ba3e3dde4b376e1b5"}
Oct 30 01:16:39.951334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:39.951206    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd" podStartSLOduration=0.951167337 podStartE2EDuration="951.167337ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:16:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:16:39.950311267 +0000 UTC m=+2242.053065237" watchObservedRunningTime="2026-10-30 01:16:39.951167337 +0000 UTC m=+2242.053921304"
Oct 30 01:16:40.372353 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:40.372310    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:41.777660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.777618    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"]
Oct 30 01:16:41.782365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.782337    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.797960 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.797924    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"]
Oct 30 01:16:41.806860 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.806823    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"sub-test-model-283eb6-kserve-self-signed-certs\""
Oct 30 01:16:41.807041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.806937    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"default-dockercfg-r6w79\""
Oct 30 01:16:41.818118 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.818089    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"kube-root-ca.crt\""
Oct 30 01:16:41.818118 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.818088    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w0-283eb6\"/\"openshift-service-ca.crt\""
Oct 30 01:16:41.881658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.881618    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-home\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.881868 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.881787    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747a8df2-d40a-401f-9d18-9acbc47d74f1-tls-certs\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.881868 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.881840    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-dshm\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.881951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.881869    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pspjs\" (UniqueName: \"kubernetes.io/projected/747a8df2-d40a-401f-9d18-9acbc47d74f1-kube-api-access-pspjs\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.882038 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.882018    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-model-cache\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.882079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.882050    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-tmp-dir\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983290    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747a8df2-d40a-401f-9d18-9acbc47d74f1-tls-certs\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983335    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-dshm\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983370    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pspjs\" (UniqueName: \"kubernetes.io/projected/747a8df2-d40a-401f-9d18-9acbc47d74f1-kube-api-access-pspjs\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983494    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-model-cache\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983518    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-tmp-dir\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983749 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983650    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-home\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983916    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-model-cache\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.983989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983952    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-tmp-dir\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.984109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.983986    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-home\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.985977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.985950    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-dshm\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:41.986145 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:41.986125    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747a8df2-d40a-401f-9d18-9acbc47d74f1-tls-certs\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:42.009405 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:42.009371    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pspjs\" (UniqueName: \"kubernetes.io/projected/747a8df2-d40a-401f-9d18-9acbc47d74f1-kube-api-access-pspjs\") pod \"sub-test-model-283eb6-kserve-5899f749cc-clcl6\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") " pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:42.101622 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:42.101581    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:42.414018 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:16:42.413983    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod747a8df2_d40a_401f_9d18_9acbc47d74f1.slice/crio-66973dc77430e44a815ff315e1726229bec811d9837f92a18e35665710f8e175 WatchSource:0}: Error finding container 66973dc77430e44a815ff315e1726229bec811d9837f92a18e35665710f8e175: Status 404 returned error can't find the container with id 66973dc77430e44a815ff315e1726229bec811d9837f92a18e35665710f8e175
Oct 30 01:16:42.414470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:42.414445    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"]
Oct 30 01:16:42.932797 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:42.932751    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6" event={"ID":"747a8df2-d40a-401f-9d18-9acbc47d74f1","Type":"ContainerStarted","Data":"329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2"}
Oct 30 01:16:42.933274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:42.932806    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6" event={"ID":"747a8df2-d40a-401f-9d18-9acbc47d74f1","Type":"ContainerStarted","Data":"66973dc77430e44a815ff315e1726229bec811d9837f92a18e35665710f8e175"}
Oct 30 01:16:42.985240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:42.985163    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6" podStartSLOduration=1.9851429029999998 podStartE2EDuration="1.985142903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:16:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:16:42.982381945 +0000 UTC m=+2245.085135916" watchObservedRunningTime="2026-10-30 01:16:42.985142903 +0000 UTC m=+2245.087896874"
Oct 30 01:16:43.103531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:43.102641    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:49.372576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:49.372527    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:49.386086 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:49.386052    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:49.997455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:49.997422    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:16:52.102042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:52.102002    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:52.114815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:52.114784    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:53.019723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:16:53.019682    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:16:57.912903 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:16:57.912862    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:17:13.132259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:13.132223    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"]
Oct 30 01:17:27.956333 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:17:27.956292    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:17:28.155084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.155014    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr" podUID="75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" containerName="main" containerID="cri-o://a23752131c3b7a5e4545b5557ff6ffba9f3f33631b5100a0aded9d764cd925c9" gracePeriod=45
Oct 30 01:17:28.286917 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.286885    6011 generic.go:356] "Generic (PLEG): container finished" podID="75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" containerID="a23752131c3b7a5e4545b5557ff6ffba9f3f33631b5100a0aded9d764cd925c9" exitCode=0
Oct 30 01:17:28.287137 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.286946    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr" event={"ID":"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9","Type":"ContainerDied","Data":"a23752131c3b7a5e4545b5557ff6ffba9f3f33631b5100a0aded9d764cd925c9"}
Oct 30 01:17:28.470229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.470202    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:17:28.603696 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.603663    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-model-cache\") pod \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") "
Oct 30 01:17:28.603885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.603711    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tmp-dir\") pod \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") "
Oct 30 01:17:28.603885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.603871    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tls-certs\" (UniqueName: \"kubernetes.io/secret/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tls-certs\") pod \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") "
Oct 30 01:17:28.603990 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.603899    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-kube-api-access-wqvvr\" (UniqueName: \"kubernetes.io/projected/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-kube-api-access-wqvvr\") pod \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") "
Oct 30 01:17:28.603990 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.603944    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-dshm\") pod \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") "
Oct 30 01:17:28.603990 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.603964    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-home\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-home\") pod \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\" (UID: \"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9\") "
Oct 30 01:17:28.603990 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.603944    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-model-cache" pod "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" (UID: "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:28.604236 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.604012    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tmp-dir" pod "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" (UID: "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:28.604329 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.604293    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-home" pod "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" (UID: "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:28.604611 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.604582    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:28.604611 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.604612    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:28.604611 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.604624    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:28.606496 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.606474    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-kube-api-access-wqvvr" pod "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" (UID: "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9"). InnerVolumeSpecName "kube-api-access-wqvvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:17:28.606595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.606487    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-dshm" pod "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" (UID: "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:28.606595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.606558    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tls-certs" pod "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" (UID: "75ccd50e-3228-4a9a-9597-f37ebc2e8ec9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:17:28.705864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.705824    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:28.705864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.705859    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqvvr\" (UniqueName: \"kubernetes.io/projected/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-kube-api-access-wqvvr\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:28.705864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:28.705870    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:29.294974 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.294944    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"
Oct 30 01:17:29.294974 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.294964    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr" event={"ID":"75ccd50e-3228-4a9a-9597-f37ebc2e8ec9","Type":"ContainerDied","Data":"85d052e805795cd4c44e837bc62c8f60ef47a9f0ac14f966b4b99f2fa6f96fcd"}
Oct 30 01:17:29.295552 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.295017    6011 scope.go:122] "RemoveContainer" containerID="a23752131c3b7a5e4545b5557ff6ffba9f3f33631b5100a0aded9d764cd925c9"
Oct 30 01:17:29.351041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.351007    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"]
Oct 30 01:17:29.363997 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.363965    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w5-bdf6f3/auth-test-model-bdf6f3-kserve-854c97475c-hfpmr"]
Oct 30 01:17:29.634570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.634461    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"]
Oct 30 01:17:29.635950 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.635914    6011 cpu_manager.go:414] "containerMap: removing container" podUID="75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" containerName="main"
Oct 30 01:17:29.636214 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.636196    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" containerName="main"
Oct 30 01:17:29.636525 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.636510    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" containerName="main"
Oct 30 01:17:29.642225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.642201    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.656076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.656046    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"]
Oct 30 01:17:29.665286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.665241    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w6-42d56b-openshift-default-dockercfg-7msn4\""
Oct 30 01:17:29.721127 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721086    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-data\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721127 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721129    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-podinfo\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721372 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721299    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-envoy\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721372 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721332    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-credential-socket\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721443 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721429    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-certs\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721478 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721449    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istiod-ca-cert\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721580 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721559    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-socket\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721597    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-token\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.721709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.721691    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6l5k\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-kube-api-access-c6l5k\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.822971    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-certs\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823014    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istiod-ca-cert\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823342 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823101    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-socket\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823342 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823143    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-token\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823342 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823264    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6l5k\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-kube-api-access-c6l5k\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823508 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823405    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-data\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823508 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823437    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-podinfo\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823508 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823446    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-certs\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823513    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-socket\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823595    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-envoy\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823626    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-credential-socket\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823793    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-data\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.823900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.823815    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istiod-ca-cert\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.824046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.824018    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-credential-socket\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.825963 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.825941    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-envoy\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.826212 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.826191    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-podinfo\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.843783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.843755    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-token\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.843899 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.843854    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6l5k\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-kube-api-access-c6l5k\") pod \"e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") " pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:29.960131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:29.960022    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:30.026663 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:30.026619    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75ccd50e-3228-4a9a-9597-f37ebc2e8ec9" path="/var/lib/kubelet/pods/75ccd50e-3228-4a9a-9597-f37ebc2e8ec9/volumes"
Oct 30 01:17:30.297745 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:17:30.297715    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138f9a89_b2d0_48ef_80a5_2517d6074e8b.slice/crio-26c164f22eb5a89023ac61c4436834d035269c094643fe71bcdbb8a98636cb8f WatchSource:0}: Error finding container 26c164f22eb5a89023ac61c4436834d035269c094643fe71bcdbb8a98636cb8f: Status 404 returned error can't find the container with id 26c164f22eb5a89023ac61c4436834d035269c094643fe71bcdbb8a98636cb8f
Oct 30 01:17:30.298322 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:30.298296    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"]
Oct 30 01:17:30.300856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:30.300817    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:17:30.300952 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:30.300905    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:17:30.301035 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:30.300954    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:17:31.212240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.201230    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"]
Oct 30 01:17:31.212240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.201516    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn" podUID="d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" containerName="maas-api" containerID="cri-o://4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd" gracePeriod=30
Oct 30 01:17:31.313112 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.313065    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" event={"ID":"138f9a89-b2d0-48ef-80a5-2517d6074e8b","Type":"ContainerStarted","Data":"f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd"}
Oct 30 01:17:31.313524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.313140    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" event={"ID":"138f9a89-b2d0-48ef-80a5-2517d6074e8b","Type":"ContainerStarted","Data":"26c164f22eb5a89023ac61c4436834d035269c094643fe71bcdbb8a98636cb8f"}
Oct 30 01:17:31.366514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.366435    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" podStartSLOduration=2.366414592 podStartE2EDuration="2.366414592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:17:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:17:31.365728974 +0000 UTC m=+2293.468482947" watchObservedRunningTime="2026-10-30 01:17:31.366414592 +0000 UTC m=+2293.469168564"
Oct 30 01:17:31.620230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.619906    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:17:31.705136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.705087    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-maas-api-tls\") pod \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") "
Oct 30 01:17:31.705417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.705151    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-kube-api-access-rtrs6\" (UniqueName: \"kubernetes.io/projected/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-kube-api-access-rtrs6\") pod \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") "
Oct 30 01:17:31.705417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.705341    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-openshift-service-ca\") pod \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") "
Oct 30 01:17:31.705548 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.705525    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-metrics-certs\") pod \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\" (UID: \"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0\") "
Oct 30 01:17:31.711231 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.708001    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-openshift-service-ca" pod "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" (UID: "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:17:31.711231 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.710930    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-maas-api-tls" pod "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" (UID: "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:17:31.711231 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.711144    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-kube-api-access-rtrs6" pod "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" (UID: "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0"). InnerVolumeSpecName "kube-api-access-rtrs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:17:31.713152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.713125    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-metrics-certs" pod "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" (UID: "d7c8458a-e080-49aa-b8ac-1cc2fab48bc0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:17:31.808471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.808377    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:31.808471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.808421    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:31.808471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.808435    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:31.808471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.808448    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtrs6\" (UniqueName: \"kubernetes.io/projected/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0-kube-api-access-rtrs6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:31.967203 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.966423    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:31.968121 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.968086    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 01:17:31.968304 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:31.968145    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 01:17:32.321411 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.321373    6011 generic.go:356] "Generic (PLEG): container finished" podID="d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" containerID="4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd" exitCode=0
Oct 30 01:17:32.321915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.321423    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"
Oct 30 01:17:32.321915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.321460    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn" event={"ID":"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0","Type":"ContainerDied","Data":"4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd"}
Oct 30 01:17:32.321915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.321520    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn" event={"ID":"d7c8458a-e080-49aa-b8ac-1cc2fab48bc0","Type":"ContainerDied","Data":"c265bf98086903fe36ad5b5c747b095c37c4b85261d273e737334983246cc928"}
Oct 30 01:17:32.321915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.321544    6011 scope.go:122] "RemoveContainer" containerID="4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd"
Oct 30 01:17:32.332488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.332469    6011 scope.go:122] "RemoveContainer" containerID="4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd"
Oct 30 01:17:32.332806 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:17:32.332782    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd\": container with ID starting with 4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd not found: ID does not exist" containerID="4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd"
Oct 30 01:17:32.332871 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.332817    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd"} err="failed to get container status \"4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd\": rpc error: code = NotFound desc = could not find container \"4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd\": container with ID starting with 4955d8f33d8499e62fed9f13c0d6f82ff996cb2c3fb33ae89e0578206be04efd not found: ID does not exist"
Oct 30 01:17:32.378442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.378407    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"]
Oct 30 01:17:32.391762 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.391728    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-95fe33-6c5db9ddf9-w5ffn"]
Oct 30 01:17:32.961685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.961226    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 01:17:32.961685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:32.961292    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 01:17:33.964207 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:33.962827    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 01:17:33.964207 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:33.962888    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 01:17:34.034311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:34.034273    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" path="/var/lib/kubelet/pods/d7c8458a-e080-49aa-b8ac-1cc2fab48bc0/volumes"
Oct 30 01:17:34.964720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:34.964679    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 01:17:34.965154 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:34.964746    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 01:17:35.572418 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:35.572372    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"]
Oct 30 01:17:35.572685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:35.572648    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" containerID="cri-o://5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b" gracePeriod=30
Oct 30 01:17:35.964460 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:35.964330    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused" start-of-body=
Oct 30 01:17:35.964460 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:35.964439    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.132:15021/healthz/ready\": dial tcp 10.134.0.132:15021: connect: connection refused"
Oct 30 01:17:36.981618 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:36.981583    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:36.982142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:36.982013    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:36.983459 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:36.983431    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:17:40.196821 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.196773    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-cxjtp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:17:40.197268 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.196857    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:17:40.859681 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.859614    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 01:17:40.980586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980548    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-socket\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.980765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980607    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d68c4f9-2522-49ac-8570-c45a39dcce25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d68c4f9-2522-49ac-8570-c45a39dcce25-istiod-ca-cert\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.980765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980713    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-envoy\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.980765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980750    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-podinfo\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.980917 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980870    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-token\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-token\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.980917 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980874    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-socket" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:40.981030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980925    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-data\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.981030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.980968    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-credential-socket\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.981143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.981074    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d68c4f9-2522-49ac-8570-c45a39dcce25-istiod-ca-cert" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:17:40.981143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.981092    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-certs\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.981143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.981131    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-kube-api-access-xsd9h\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-kube-api-access-xsd9h\") pod \"3d68c4f9-2522-49ac-8570-c45a39dcce25\" (UID: \"3d68c4f9-2522-49ac-8570-c45a39dcce25\") "
Oct 30 01:17:40.981363 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.981310    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-credential-socket" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:40.981363 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.981337    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-data" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:40.981481 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.981457    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-certs" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:40.982052 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.982023    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:40.982155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.982059    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d68c4f9-2522-49ac-8570-c45a39dcce25-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:40.982155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.982077    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:40.982155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.982091    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:40.982155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.982106    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:40.983271 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.983239    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-token" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:17:40.983387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.983311    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-envoy" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:17:40.983387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.983346    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-podinfo" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:17:40.983578 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:40.983561    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-kube-api-access-xsd9h" pod "3d68c4f9-2522-49ac-8570-c45a39dcce25" (UID: "3d68c4f9-2522-49ac-8570-c45a39dcce25"). InnerVolumeSpecName "kube-api-access-xsd9h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:17:41.083865 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.083834    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:41.084054 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.083882    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:41.084054 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.083900    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:41.084054 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.083914    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsd9h\" (UniqueName: \"kubernetes.io/projected/3d68c4f9-2522-49ac-8570-c45a39dcce25-kube-api-access-xsd9h\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:17:41.422643 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.422603    6011 generic.go:356] "Generic (PLEG): container finished" podID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerID="5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b" exitCode=0
Oct 30 01:17:41.423085 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.422662    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"
Oct 30 01:17:41.423085 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.422682    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" event={"ID":"3d68c4f9-2522-49ac-8570-c45a39dcce25","Type":"ContainerDied","Data":"5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b"}
Oct 30 01:17:41.423085 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.422740    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp" event={"ID":"3d68c4f9-2522-49ac-8570-c45a39dcce25","Type":"ContainerDied","Data":"36722e1ee6102bc6b3eec279f2558c40a2113ff5f8ecb6411a38954278c0e604"}
Oct 30 01:17:41.423085 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.422764    6011 scope.go:122] "RemoveContainer" containerID="5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b"
Oct 30 01:17:41.432135 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.432106    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"]
Oct 30 01:17:41.433520 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.433281    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" containerName="maas-api"
Oct 30 01:17:41.433520 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.433304    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" containerName="maas-api"
Oct 30 01:17:41.433520 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.433366    6011 cpu_manager.go:414] "containerMap: removing container" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy"
Oct 30 01:17:41.433520 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.433375    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy"
Oct 30 01:17:41.433836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.433575    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" containerName="istio-proxy"
Oct 30 01:17:41.433836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.433604    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7c8458a-e080-49aa-b8ac-1cc2fab48bc0" containerName="maas-api"
Oct 30 01:17:41.438900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.438878    6011 scope.go:122] "RemoveContainer" containerID="5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b"
Oct 30 01:17:41.439578 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:17:41.439509    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b\": container with ID starting with 5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b not found: ID does not exist" containerID="5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b"
Oct 30 01:17:41.439578 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.439546    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b"} err="failed to get container status \"5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b\": rpc error: code = NotFound desc = could not find container \"5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b\": container with ID starting with 5eeb5568eba1b418160a50d1cfdba2f2663b97bf4503a7cac2a3edce7e29db8b not found: ID does not exist"
Oct 30 01:17:41.439828 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.439801    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.453598 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.453573    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"]
Oct 30 01:17:41.457498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.457473    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-cb31e3\"/\"kube-root-ca.crt\""
Oct 30 01:17:41.457498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.457489    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w1-cb31e3\"/\"openshift-service-ca.crt\""
Oct 30 01:17:41.457657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.457526    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-cb31e3\"/\"default-dockercfg-f47j9\""
Oct 30 01:17:41.461530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.461497    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w1-cb31e3\"/\"auth-test-model-cb31e3-kserve-self-signed-certs\""
Oct 30 01:17:41.486407 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.486373    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"]
Oct 30 01:17:41.492115 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.492082    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-cxjtp"]
Oct 30 01:17:41.589672 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.589638    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/465bd602-695a-45e6-b8ab-0aac9f2b2846-tls-certs\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.589855 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.589702    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-model-cache\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.589855 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.589819    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4kg9\" (UniqueName: \"kubernetes.io/projected/465bd602-695a-45e6-b8ab-0aac9f2b2846-kube-api-access-x4kg9\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.589855 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.589844    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-tmp-dir\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.589972 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.589958    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-home\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.590165 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.590141    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-dshm\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.693529    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-dshm\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.693634    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/465bd602-695a-45e6-b8ab-0aac9f2b2846-tls-certs\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.693692    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-model-cache\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.693752    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4kg9\" (UniqueName: \"kubernetes.io/projected/465bd602-695a-45e6-b8ab-0aac9f2b2846-kube-api-access-x4kg9\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.693798    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-tmp-dir\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.693879    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-home\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.694382    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-home\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.694695    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-model-cache\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.702748    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-tmp-dir\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.703842 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.703370    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-dshm\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.705693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.705662    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/465bd602-695a-45e6-b8ab-0aac9f2b2846-tls-certs\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.717936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.717903    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4kg9\" (UniqueName: \"kubernetes.io/projected/465bd602-695a-45e6-b8ab-0aac9f2b2846-kube-api-access-x4kg9\") pod \"auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") " pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:41.763408 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:41.763374    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:42.023343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:42.023264    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d68c4f9-2522-49ac-8570-c45a39dcce25" path="/var/lib/kubelet/pods/3d68c4f9-2522-49ac-8570-c45a39dcce25/volumes"
Oct 30 01:17:42.087025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:42.086996    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"]
Oct 30 01:17:42.433154 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:42.433113    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl" event={"ID":"465bd602-695a-45e6-b8ab-0aac9f2b2846","Type":"ContainerStarted","Data":"76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073"}
Oct 30 01:17:42.433674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:42.433164    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl" event={"ID":"465bd602-695a-45e6-b8ab-0aac9f2b2846","Type":"ContainerStarted","Data":"e2c60e0c169e361b1f432a4b228d7501e9c62113385cfad3e44dcfce1a0201e6"}
Oct 30 01:17:42.474034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:42.473981    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl" podStartSLOduration=1.473964455 podStartE2EDuration="1.473964455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:17:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:17:42.472913648 +0000 UTC m=+2304.575667618" watchObservedRunningTime="2026-10-30 01:17:42.473964455 +0000 UTC m=+2304.576718425"
Oct 30 01:17:42.763869 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:42.763780    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:47.825357 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.825311    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"]
Oct 30 01:17:47.829995 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.829965    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.858269 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.858228    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f461c3-serving-cert\""
Oct 30 01:17:47.864218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.864119    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"]
Oct 30 01:17:47.887743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.887700    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg8wj\" (UniqueName: \"kubernetes.io/projected/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-kube-api-access-tg8wj\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.887971 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.887759    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-metrics-certs\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.888333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.888296    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.888723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.888689    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.989820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.989777    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.989991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.989951    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.990096 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.990077    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg8wj\" (UniqueName: \"kubernetes.io/projected/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-kube-api-access-tg8wj\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.990157 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.990111    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-metrics-certs\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.990684 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.990642    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.992866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.992838    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-metrics-certs\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:47.993036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:47.993016    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:48.010977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:48.010939    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg8wj\" (UniqueName: \"kubernetes.io/projected/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-kube-api-access-tg8wj\") pod \"maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:48.146410 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:48.146317    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:48.458089 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:48.458053    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"]
Oct 30 01:17:48.458609 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:17:48.458585    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f61aac_9ddd_47bc_86a8_7fd100a2e19d.slice/crio-c6af07b7a1a7742a912fb8376e329ee6c23710bb355aafe9a349b16aed2bbfb6 WatchSource:0}: Error finding container c6af07b7a1a7742a912fb8376e329ee6c23710bb355aafe9a349b16aed2bbfb6: Status 404 returned error can't find the container with id c6af07b7a1a7742a912fb8376e329ee6c23710bb355aafe9a349b16aed2bbfb6
Oct 30 01:17:48.460140 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:48.460120    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:17:48.489779 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:48.489735    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j" event={"ID":"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d","Type":"ContainerStarted","Data":"c6af07b7a1a7742a912fb8376e329ee6c23710bb355aafe9a349b16aed2bbfb6"}
Oct 30 01:17:49.498530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:49.498484    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j" event={"ID":"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d","Type":"ContainerStarted","Data":"9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78"}
Oct 30 01:17:49.499032 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:49.498769    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:49.540310 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:49.540241    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j" podStartSLOduration=2.137279952 podStartE2EDuration="2.540227619s" totalImagesPullingTime="402.947667ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:17:49.538981069 +0000 UTC m=+2311.641735038" watchObservedRunningTime="2026-10-30 01:17:49.540227619 +0000 UTC m=+2311.642981587"
Oct 30 01:17:51.764460 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:51.764413    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:51.777698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:51.777654    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:52.538323 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:52.538293    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:17:54.034942 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.034908    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"]
Oct 30 01:17:54.039830 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.039808    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.057010 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.056977    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"]
Oct 30 01:17:54.069223 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.065688    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-1075cb-serving-cert\""
Oct 30 01:17:54.082582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.082546    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.082883 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.082829    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98692fcb-68d5-455c-8379-46033ebdf716-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.083058 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.082955    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65vd4\" (UniqueName: \"kubernetes.io/projected/98692fcb-68d5-455c-8379-46033ebdf716-kube-api-access-65vd4\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.083480 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.083455    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-metrics-certs\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.184912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.184874    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.184912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.184921    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98692fcb-68d5-455c-8379-46033ebdf716-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.185146 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.184962    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65vd4\" (UniqueName: \"kubernetes.io/projected/98692fcb-68d5-455c-8379-46033ebdf716-kube-api-access-65vd4\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.185233 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.185165    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-metrics-certs\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.185732 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.185709    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98692fcb-68d5-455c-8379-46033ebdf716-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.187656 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.187633    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-maas-api-tls\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.187783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.187759    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-metrics-certs\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.206687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.206653    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65vd4\" (UniqueName: \"kubernetes.io/projected/98692fcb-68d5-455c-8379-46033ebdf716-kube-api-access-65vd4\") pod \"maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.357033 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.356926    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:54.663642 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:17:54.663603    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98692fcb_68d5_455c_8379_46033ebdf716.slice/crio-b38e6ae53915e64926b446e40ba577e6613c2699528af796a5fffdc14fb2965b WatchSource:0}: Error finding container b38e6ae53915e64926b446e40ba577e6613c2699528af796a5fffdc14fb2965b: Status 404 returned error can't find the container with id b38e6ae53915e64926b446e40ba577e6613c2699528af796a5fffdc14fb2965b
Oct 30 01:17:54.663895 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:54.663872    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"]
Oct 30 01:17:55.512236 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:55.512148    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:17:55.554272 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:55.554234    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw" event={"ID":"98692fcb-68d5-455c-8379-46033ebdf716","Type":"ContainerStarted","Data":"1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179"}
Oct 30 01:17:55.554500 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:55.554301    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw" event={"ID":"98692fcb-68d5-455c-8379-46033ebdf716","Type":"ContainerStarted","Data":"b38e6ae53915e64926b446e40ba577e6613c2699528af796a5fffdc14fb2965b"}
Oct 30 01:17:55.554500 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:55.554459    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:17:55.599400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:17:55.599327    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw" podStartSLOduration=1.192111941 podStartE2EDuration="1.599304854s" totalImagesPullingTime="407.192913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:17:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:17:55.594682687 +0000 UTC m=+2317.697436659" watchObservedRunningTime="2026-10-30 01:17:55.599304854 +0000 UTC m=+2317.702058827"
Oct 30 01:17:57.915853 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:17:57.915808    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:18:01.568886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:18:01.568854    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:18:27.928640 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:18:27.928542    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:18:57.917116 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:18:57.917078    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:19:07.705671 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.705601    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"]
Oct 30 01:19:07.711874 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.711842    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:07.712942 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.712904    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74046b45-fc09-463e-8034-7e0797071d15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:07.716504 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.716467    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6d4h\" (UniqueName: \"kubernetes.io/projected/74046b45-fc09-463e-8034-7e0797071d15-kube-api-access-h6d4h\") pod \"payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:07.768417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.768384    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"]
Oct 30 01:19:07.768665 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.768511    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f461c3\""
Oct 30 01:19:07.768779 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.768654    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f461c3-dockercfg-582hf\""
Oct 30 01:19:07.817813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.817775    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6d4h\" (UniqueName: \"kubernetes.io/projected/74046b45-fc09-463e-8034-7e0797071d15-kube-api-access-h6d4h\") pod \"payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:07.818651 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.818621    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74046b45-fc09-463e-8034-7e0797071d15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:07.819568 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.819541    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74046b45-fc09-463e-8034-7e0797071d15-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:07.839313 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.839282    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6d4h\" (UniqueName: \"kubernetes.io/projected/74046b45-fc09-463e-8034-7e0797071d15-kube-api-access-h6d4h\") pod \"payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:07.866553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.866515    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"]
Oct 30 01:19:07.871404 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.871374    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:07.892822 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.892786    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"]
Oct 30 01:19:07.920043 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.920003    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:07.920354 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:07.920331    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxgsm\" (UniqueName: \"kubernetes.io/projected/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-kube-api-access-bxgsm\") pod \"payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:08.021779 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.021691    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxgsm\" (UniqueName: \"kubernetes.io/projected/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-kube-api-access-bxgsm\") pod \"payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:08.021966 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.021800    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:08.022588 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.022565    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:08.042376 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.042347    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxgsm\" (UniqueName: \"kubernetes.io/projected/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-kube-api-access-bxgsm\") pod \"payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:08.099165 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.099119    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:08.190255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.188545    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:08.419708 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.419670    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"]
Oct 30 01:19:08.420033 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:08.420007    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74046b45_fc09_463e_8034_7e0797071d15.slice/crio-4d4d228fa5c10b23f38d96bd5e2bf13a82ab012860a0f48ad37454bca6300f09 WatchSource:0}: Error finding container 4d4d228fa5c10b23f38d96bd5e2bf13a82ab012860a0f48ad37454bca6300f09: Status 404 returned error can't find the container with id 4d4d228fa5c10b23f38d96bd5e2bf13a82ab012860a0f48ad37454bca6300f09
Oct 30 01:19:08.548499 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:08.548468    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fafbcc8_ddc6_4fd7_bd0a_f1fb67657f2d.slice/crio-0ac6ad188be479189b22485efec3cbcb1548ad22f0cd76388155e23d93976338 WatchSource:0}: Error finding container 0ac6ad188be479189b22485efec3cbcb1548ad22f0cd76388155e23d93976338: Status 404 returned error can't find the container with id 0ac6ad188be479189b22485efec3cbcb1548ad22f0cd76388155e23d93976338
Oct 30 01:19:08.548819 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:08.548793    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"]
Oct 30 01:19:09.291721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:09.291657    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4" event={"ID":"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d","Type":"ContainerStarted","Data":"0ac6ad188be479189b22485efec3cbcb1548ad22f0cd76388155e23d93976338"}
Oct 30 01:19:09.295395 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:09.295338    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf" event={"ID":"74046b45-fc09-463e-8034-7e0797071d15","Type":"ContainerStarted","Data":"4d4d228fa5c10b23f38d96bd5e2bf13a82ab012860a0f48ad37454bca6300f09"}
Oct 30 01:19:10.468292 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.465037    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"]
Oct 30 01:19:10.475882 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.475845    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.491020 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.490106    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"]
Oct 30 01:19:10.546016 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.545983    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w4-60f38f-openshift-default-dockercfg-tvpjg\""
Oct 30 01:19:10.573681 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.573646    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.573875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.573772    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.573875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.573825    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.573875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.573846    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jfcm\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-kube-api-access-9jfcm\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.574059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.573909    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.574059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.574011    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.574190 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.574070    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.574256 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.574210    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2805fa76-0d4d-4185-aacb-b7b5923985a9-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.574256 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.574242    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.675399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675301    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.675399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675371    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.675649 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675418    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.675649 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675444    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jfcm\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-kube-api-access-9jfcm\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.675649 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675483    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.675649 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675509    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.675649 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675574    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.676006 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675664    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2805fa76-0d4d-4185-aacb-b7b5923985a9-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.676006 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675695    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.676006 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675874    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.676006 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.675939    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.676384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.676224    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.677134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.677104    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2805fa76-0d4d-4185-aacb-b7b5923985a9-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.677468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.677445    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.678668 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.678642    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.678764 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.678731    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.699155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.699102    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.699354 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.699275    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jfcm\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-kube-api-access-9jfcm\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:10.839146 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:10.839086    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:11.165030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.164992    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"]
Oct 30 01:19:11.165374 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:11.165347    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2805fa76_0d4d_4185_aacb_b7b5923985a9.slice/crio-02f221209e85e76725f91521bcac79e83a8ae25ae09c3ce749f7e6e0ca829bcd WatchSource:0}: Error finding container 02f221209e85e76725f91521bcac79e83a8ae25ae09c3ce749f7e6e0ca829bcd: Status 404 returned error can't find the container with id 02f221209e85e76725f91521bcac79e83a8ae25ae09c3ce749f7e6e0ca829bcd
Oct 30 01:19:11.168550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.168510    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:11.168650 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.168582    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:11.168650 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.168612    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:11.315859 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.315812    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" event={"ID":"2805fa76-0d4d-4185-aacb-b7b5923985a9","Type":"ContainerStarted","Data":"ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff"}
Oct 30 01:19:11.316076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.315909    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" event={"ID":"2805fa76-0d4d-4185-aacb-b7b5923985a9","Type":"ContainerStarted","Data":"02f221209e85e76725f91521bcac79e83a8ae25ae09c3ce749f7e6e0ca829bcd"}
Oct 30 01:19:11.317813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.317775    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4" event={"ID":"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d","Type":"ContainerStarted","Data":"57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961"}
Oct 30 01:19:11.318255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.317852    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:11.320753 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.320729    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf" event={"ID":"74046b45-fc09-463e-8034-7e0797071d15","Type":"ContainerStarted","Data":"389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3"}
Oct 30 01:19:11.320922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.320812    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:11.361336 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.361273    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" podStartSLOduration=1.361255949 podStartE2EDuration="1.361255949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:11.358714563 +0000 UTC m=+2393.461468559" watchObservedRunningTime="2026-10-30 01:19:11.361255949 +0000 UTC m=+2393.464009973"
Oct 30 01:19:11.404863 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.404793    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf" podStartSLOduration=2.478541732 podStartE2EDuration="4.404773968s" totalImagesPullingTime="1.926232236s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:11.400302397 +0000 UTC m=+2393.503056373" watchObservedRunningTime="2026-10-30 01:19:11.404773968 +0000 UTC m=+2393.507527938"
Oct 30 01:19:11.442079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.441965    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4" podStartSLOduration=2.538608487 podStartE2EDuration="4.441951052s" totalImagesPullingTime="1.903342565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:11.440922284 +0000 UTC m=+2393.543676252" watchObservedRunningTime="2026-10-30 01:19:11.441951052 +0000 UTC m=+2393.544705021"
Oct 30 01:19:11.839743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:11.839700    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:12.864379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:12.864329    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 01:19:12.864944 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:12.864393    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 01:19:13.859202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:13.849610    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 01:19:13.859202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:13.849678    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 01:19:14.859203 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:14.854642    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 01:19:14.859203 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:14.854713    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 01:19:15.624245 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.601237    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"]
Oct 30 01:19:15.627288 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.627256    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:15.635692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.629420    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"]
Oct 30 01:19:15.651467 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.651149    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-1075cb\""
Oct 30 01:19:15.659217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.658445    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-1075cb-dockercfg-rh4dd\""
Oct 30 01:19:15.669794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.669350    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:15.669794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.669536    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-596d4\" (UniqueName: \"kubernetes.io/projected/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-kube-api-access-596d4\") pod \"payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:15.771269 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.771223    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:15.772009 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.771981    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-596d4\" (UniqueName: \"kubernetes.io/projected/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-kube-api-access-596d4\") pod \"payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:15.773266 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.773236    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:15.793196 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.793135    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-596d4\" (UniqueName: \"kubernetes.io/projected/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-kube-api-access-596d4\") pod \"payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:15.864644 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.864604    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 01:19:15.865235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.864672    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 01:19:15.969734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:15.969633    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:16.275202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:16.275146    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"]
Oct 30 01:19:16.275320 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:16.275270    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf27f286f_4c8a_4a2b_a99f_c85119b8ebd5.slice/crio-727eb58e256f89f3ac2452f54eb3b4f67c3f1de535bd4e719ccaeb719cacb163 WatchSource:0}: Error finding container 727eb58e256f89f3ac2452f54eb3b4f67c3f1de535bd4e719ccaeb719cacb163: Status 404 returned error can't find the container with id 727eb58e256f89f3ac2452f54eb3b4f67c3f1de535bd4e719ccaeb719cacb163
Oct 30 01:19:16.417332 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:16.417281    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7" event={"ID":"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5","Type":"ContainerStarted","Data":"e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95"}
Oct 30 01:19:16.418999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:16.417355    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7" event={"ID":"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5","Type":"ContainerStarted","Data":"727eb58e256f89f3ac2452f54eb3b4f67c3f1de535bd4e719ccaeb719cacb163"}
Oct 30 01:19:16.418999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:16.418598    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:16.469220 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:16.465553    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7" podStartSLOduration=1.465533052 podStartE2EDuration="1.465533052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:16.46452612 +0000 UTC m=+2398.567280149" watchObservedRunningTime="2026-10-30 01:19:16.465533052 +0000 UTC m=+2398.568287024"
Oct 30 01:19:16.865385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:16.865342    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 01:19:16.866038 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:16.865988    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 01:19:17.138102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:17.137670    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"]
Oct 30 01:19:17.980330 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:17.980297    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:17.980977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:17.980955    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:17.981840 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:17.981819    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:19:22.332073 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:22.332035    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:19:22.332503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:22.332243    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:19:26.122099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.122055    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"]
Oct 30 01:19:26.127402 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.127339    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.168072 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.167688    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"]
Oct 30 01:19:26.171853 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.171633    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w0-283eb6-openshift-default-dockercfg-kw2jr\""
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260107    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-token\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260412    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-credential-socket\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260460    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-podinfo\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260513    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-data\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260541    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istiod-ca-cert\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260613    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-certs\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260693    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-socket\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260803    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-envoy\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.261202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.260832    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkbwp\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-kube-api-access-tkbwp\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.362782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.362745    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-token\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.362782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.362793    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-credential-socket\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.362820    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-podinfo\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.362869    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-data\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.362895    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istiod-ca-cert\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.363053    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-certs\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.363239    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-socket\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.363373    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-envoy\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.363401    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkbwp\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-kube-api-access-tkbwp\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.363728 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.363526    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-data\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.364491 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.364466    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-credential-socket\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.364684 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.364488    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-certs\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.364684 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.364567    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-socket\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.365144 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.365120    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istiod-ca-cert\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.366285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.366251    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-podinfo\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.366896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.366872    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-envoy\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.384502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.384428    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-token\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.385111 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.385084    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkbwp\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-kube-api-access-tkbwp\") pod \"e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") " pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.472698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.472662    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:26.794514 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:26.794479    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaaae85b_371c_4f62_8d80_dfaa7cb7688e.slice/crio-f94099816a74892ca11e3d4d08da20da392f8c8043d4ef7732c88232dcbae363 WatchSource:0}: Error finding container f94099816a74892ca11e3d4d08da20da392f8c8043d4ef7732c88232dcbae363: Status 404 returned error can't find the container with id f94099816a74892ca11e3d4d08da20da392f8c8043d4ef7732c88232dcbae363
Oct 30 01:19:26.794837 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.794818    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"]
Oct 30 01:19:26.797420 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.797389    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:26.797489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.797455    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:26.797533 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:26.797507    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:27.525705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:27.525670    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" event={"ID":"daaae85b-371c-4f62-8d80-dfaa7cb7688e","Type":"ContainerStarted","Data":"270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f"}
Oct 30 01:19:27.526142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:27.525725    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" event={"ID":"daaae85b-371c-4f62-8d80-dfaa7cb7688e","Type":"ContainerStarted","Data":"f94099816a74892ca11e3d4d08da20da392f8c8043d4ef7732c88232dcbae363"}
Oct 30 01:19:27.583588 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:27.583535    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podStartSLOduration=1.583520907 podStartE2EDuration="1.583520907s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:27.581852461 +0000 UTC m=+2409.684606429" watchObservedRunningTime="2026-10-30 01:19:27.583520907 +0000 UTC m=+2409.686274879"
Oct 30 01:19:27.911705 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:19:27.911667    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:19:28.466609 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.466574    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:19:28.473226 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.473200    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:28.474640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.474613    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:19:28.474738 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.474660    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:19:28.568715 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.568682    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"]
Oct 30 01:19:28.573154 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.573130    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.594457 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.594420    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-5c515e-serving-cert\""
Oct 30 01:19:28.596810 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.596778    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"]
Oct 30 01:19:28.708298 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.708261    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.708473 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.708310    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d360ea0-4cc2-4236-b5df-e3dd6369712a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.708524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.708482    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m52fd\" (UniqueName: \"kubernetes.io/projected/2d360ea0-4cc2-4236-b5df-e3dd6369712a-kube-api-access-m52fd\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.708667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.708649    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-metrics-certs\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.810471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.810433    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m52fd\" (UniqueName: \"kubernetes.io/projected/2d360ea0-4cc2-4236-b5df-e3dd6369712a-kube-api-access-m52fd\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.810631 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.810536    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-metrics-certs\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.810723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.810706    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.810790 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.810744    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d360ea0-4cc2-4236-b5df-e3dd6369712a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.811547 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.811514    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d360ea0-4cc2-4236-b5df-e3dd6369712a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.813429 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.813407    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.813515 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.813478    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-metrics-certs\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.830867 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.830832    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m52fd\" (UniqueName: \"kubernetes.io/projected/2d360ea0-4cc2-4236-b5df-e3dd6369712a-kube-api-access-m52fd\") pod \"maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:28.896782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:28.896741    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:29.268556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:29.268522    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"]
Oct 30 01:19:29.270505 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:29.270470    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d360ea0_4cc2_4236_b5df_e3dd6369712a.slice/crio-e02c0f893902a9cde1e67f138b92777f13504596fcdac0322b2410d8bca633d9 WatchSource:0}: Error finding container e02c0f893902a9cde1e67f138b92777f13504596fcdac0322b2410d8bca633d9: Status 404 returned error can't find the container with id e02c0f893902a9cde1e67f138b92777f13504596fcdac0322b2410d8bca633d9
Oct 30 01:19:29.474654 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:29.474617    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:19:29.474841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:29.474672    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:19:29.569719 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:29.569678    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr" event={"ID":"2d360ea0-4cc2-4236-b5df-e3dd6369712a","Type":"ContainerStarted","Data":"e02c0f893902a9cde1e67f138b92777f13504596fcdac0322b2410d8bca633d9"}
Oct 30 01:19:30.473916 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:30.473880    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:19:30.474132 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:30.473943    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:19:30.578599 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:30.578562    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr" event={"ID":"2d360ea0-4cc2-4236-b5df-e3dd6369712a","Type":"ContainerStarted","Data":"8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16"}
Oct 30 01:19:30.579077 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:30.578655    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:30.622038 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:30.621982    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr" podStartSLOduration=2.297546782 podStartE2EDuration="2.621965061s" totalImagesPullingTime="324.418279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:30.620775642 +0000 UTC m=+2412.723529615" watchObservedRunningTime="2026-10-30 01:19:30.621965061 +0000 UTC m=+2412.724719033"
Oct 30 01:19:31.473816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:31.473783    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:19:31.473989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:31.473834    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:19:32.197248 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.197192    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd" podUID="9b417080-16fe-4b23-b53a-4a06cc221325" containerName="main" containerID="cri-o://a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d" gracePeriod=45
Oct 30 01:19:32.475866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.475368    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 01:19:32.475866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.475430    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 01:19:32.578525 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.578496    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:19:32.589605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.589577    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9b417080-16fe-4b23-b53a-4a06cc221325-kube-api-access-54cms\" (UniqueName: \"kubernetes.io/projected/9b417080-16fe-4b23-b53a-4a06cc221325-kube-api-access-54cms\") pod \"9b417080-16fe-4b23-b53a-4a06cc221325\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") "
Oct 30 01:19:32.589772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.589632    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-dshm\") pod \"9b417080-16fe-4b23-b53a-4a06cc221325\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") "
Oct 30 01:19:32.589772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.589669    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-tmp-dir\") pod \"9b417080-16fe-4b23-b53a-4a06cc221325\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") "
Oct 30 01:19:32.589772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.589733    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9b417080-16fe-4b23-b53a-4a06cc221325-tls-certs\" (UniqueName: \"kubernetes.io/secret/9b417080-16fe-4b23-b53a-4a06cc221325-tls-certs\") pod \"9b417080-16fe-4b23-b53a-4a06cc221325\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") "
Oct 30 01:19:32.589947 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.589824    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-home\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-home\") pod \"9b417080-16fe-4b23-b53a-4a06cc221325\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") "
Oct 30 01:19:32.589947 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.589866    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-model-cache\") pod \"9b417080-16fe-4b23-b53a-4a06cc221325\" (UID: \"9b417080-16fe-4b23-b53a-4a06cc221325\") "
Oct 30 01:19:32.590367 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.590339    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-tmp-dir" pod "9b417080-16fe-4b23-b53a-4a06cc221325" (UID: "9b417080-16fe-4b23-b53a-4a06cc221325"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:32.590576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.590557    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-model-cache" pod "9b417080-16fe-4b23-b53a-4a06cc221325" (UID: "9b417080-16fe-4b23-b53a-4a06cc221325"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:32.590761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.590741    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:32.590802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.590771    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:32.591118 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.591089    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-home" pod "9b417080-16fe-4b23-b53a-4a06cc221325" (UID: "9b417080-16fe-4b23-b53a-4a06cc221325"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:32.593029 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.593006    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b417080-16fe-4b23-b53a-4a06cc221325-kube-api-access-54cms" pod "9b417080-16fe-4b23-b53a-4a06cc221325" (UID: "9b417080-16fe-4b23-b53a-4a06cc221325"). InnerVolumeSpecName "kube-api-access-54cms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:32.593101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.593029    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b417080-16fe-4b23-b53a-4a06cc221325-tls-certs" pod "9b417080-16fe-4b23-b53a-4a06cc221325" (UID: "9b417080-16fe-4b23-b53a-4a06cc221325"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:19:32.595571 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.595056    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-dshm" pod "9b417080-16fe-4b23-b53a-4a06cc221325" (UID: "9b417080-16fe-4b23-b53a-4a06cc221325"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:32.597487 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.597464    6011 generic.go:356] "Generic (PLEG): container finished" podID="9b417080-16fe-4b23-b53a-4a06cc221325" containerID="a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d" exitCode=0
Oct 30 01:19:32.597576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.597496    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd" event={"ID":"9b417080-16fe-4b23-b53a-4a06cc221325","Type":"ContainerDied","Data":"a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d"}
Oct 30 01:19:32.597576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.597522    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd" event={"ID":"9b417080-16fe-4b23-b53a-4a06cc221325","Type":"ContainerDied","Data":"69d59e29f8f0c574b02c96e5b5611c710542fd3d967a471ba3e3dde4b376e1b5"}
Oct 30 01:19:32.597576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.597537    6011 scope.go:122] "RemoveContainer" containerID="a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d"
Oct 30 01:19:32.597674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.597659    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"
Oct 30 01:19:32.630159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.630129    6011 scope.go:122] "RemoveContainer" containerID="a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d"
Oct 30 01:19:32.630535 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:19:32.630513    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d\": container with ID starting with a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d not found: ID does not exist" containerID="a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d"
Oct 30 01:19:32.630607 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.630546    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d"} err="failed to get container status \"a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d\": rpc error: code = NotFound desc = could not find container \"a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d\": container with ID starting with a9bd6452f4b37bf57ecda3456aba36b7dd98ed1e31475ecee11f1ecbd287443d not found: ID does not exist"
Oct 30 01:19:32.700993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.700744    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b417080-16fe-4b23-b53a-4a06cc221325-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:32.700993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.700782    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:32.700993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.700794    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54cms\" (UniqueName: \"kubernetes.io/projected/9b417080-16fe-4b23-b53a-4a06cc221325-kube-api-access-54cms\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:32.700993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.700806    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b417080-16fe-4b23-b53a-4a06cc221325-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:32.711865 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.711366    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"]
Oct 30 01:19:32.715576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:32.715544    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-cfe8f0/auth-test-model-cfe8f0-kserve-54ffb6668f-hvhvd"]
Oct 30 01:19:33.611198 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:33.610476    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:33.611676 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:33.611650    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:33.613046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:33.613008    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:19:34.022990 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:34.022910    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9b417080-16fe-4b23-b53a-4a06cc221325" path="/var/lib/kubelet/pods/9b417080-16fe-4b23-b53a-4a06cc221325/volumes"
Oct 30 01:19:36.591320 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:36.591287    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:19:40.549927 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.549885    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"]
Oct 30 01:19:40.550677 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.550660    6011 cpu_manager.go:414] "containerMap: removing container" podUID="9b417080-16fe-4b23-b53a-4a06cc221325" containerName="main"
Oct 30 01:19:40.550677 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.550678    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9b417080-16fe-4b23-b53a-4a06cc221325" containerName="main"
Oct 30 01:19:40.550841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.550831    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="9b417080-16fe-4b23-b53a-4a06cc221325" containerName="main"
Oct 30 01:19:40.556365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.556338    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.574139 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.574108    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"]
Oct 30 01:19:40.629626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.629586    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.629626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.629626    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.629919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.629713    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.629919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.629850    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.629919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.629911    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.630189 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.629962    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.630189 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.629991    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57slf\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-kube-api-access-57slf\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.630189 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.630027    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca96d71-9252-4b22-9061-de5085b79397-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.630358 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.630197    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca96d71-9252-4b22-9061-de5085b79397-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.731763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.731712    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.731763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.731770    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.731993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.731827    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.731993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.731915    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.731993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.731958    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732019    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732046    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57slf\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-kube-api-access-57slf\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732076    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca96d71-9252-4b22-9061-de5085b79397-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732147    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca96d71-9252-4b22-9061-de5085b79397-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732301    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732445 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732339    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732445 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732406    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732457    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.732964 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.732941    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca96d71-9252-4b22-9061-de5085b79397-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.734544 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.734518    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.735024 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.735000    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca96d71-9252-4b22-9061-de5085b79397-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.755727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.755698    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.755883 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.755834    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57slf\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-kube-api-access-57slf\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:40.875889 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:40.875804    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:41.200514 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:41.200483    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca96d71_9252_4b22_9061_de5085b79397.slice/crio-96a2a85414109282a7cc05dd61c1ba1567e0a28431704d8e5b36522897645951 WatchSource:0}: Error finding container 96a2a85414109282a7cc05dd61c1ba1567e0a28431704d8e5b36522897645951: Status 404 returned error can't find the container with id 96a2a85414109282a7cc05dd61c1ba1567e0a28431704d8e5b36522897645951
Oct 30 01:19:41.202032 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.202006    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"]
Oct 30 01:19:41.203492 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.203453    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:41.203555 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.203528    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:41.203598 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.203557    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:41.722591 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.722552    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" event={"ID":"4ca96d71-9252-4b22-9061-de5085b79397","Type":"ContainerStarted","Data":"6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc"}
Oct 30 01:19:41.723115 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.722612    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" event={"ID":"4ca96d71-9252-4b22-9061-de5085b79397","Type":"ContainerStarted","Data":"96a2a85414109282a7cc05dd61c1ba1567e0a28431704d8e5b36522897645951"}
Oct 30 01:19:41.768332 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.768270    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" podStartSLOduration=1.768253408 podStartE2EDuration="1.768253408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:41.765475769 +0000 UTC m=+2423.868229758" watchObservedRunningTime="2026-10-30 01:19:41.768253408 +0000 UTC m=+2423.871007379"
Oct 30 01:19:41.876156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:41.876117    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:42.878286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:42.878245    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 01:19:42.878724 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:42.878304    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 01:19:43.876335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:43.876300    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 01:19:43.876531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:43.876367    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 01:19:44.876792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:44.876749    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 01:19:44.877249 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:44.876836    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 01:19:45.876374 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:45.876337    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 01:19:45.876660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:45.876398    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 01:19:46.878091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:46.878050    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 01:19:46.878733 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:46.878695    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 01:19:47.882712 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:47.882676    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:47.883303 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:47.883277    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:47.884158 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:47.884138    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:19:49.321658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:49.321621    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"]
Oct 30 01:19:49.322399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:49.321937    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy" containerID="cri-o://96835eacd10338c7e7fa395ae015b61338ca5b9c829724f1e97e6f783eab410d" gracePeriod=30
Oct 30 01:19:52.650262 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:52.650214    6011 patch_prober.go:29] interesting pod/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:19:52.650709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:52.650297    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:19:54.500826 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.500787    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"]
Oct 30 01:19:54.505490 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.505470    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.523259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.523217    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"]
Oct 30 01:19:54.531597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.531556    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w4-954bf6-serving-cert\""
Oct 30 01:19:54.640891 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.640796    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.640891 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.640836    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.641142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.641037    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhvtj\" (UniqueName: \"kubernetes.io/projected/2748e587-acac-4006-a273-e9ddfdd34657-kube-api-access-dhvtj\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.641142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.641098    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2748e587-acac-4006-a273-e9ddfdd34657-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.744486 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.744220    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.744486 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.744288    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.744486 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.744462    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhvtj\" (UniqueName: \"kubernetes.io/projected/2748e587-acac-4006-a273-e9ddfdd34657-kube-api-access-dhvtj\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.744858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.744520    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2748e587-acac-4006-a273-e9ddfdd34657-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.747207 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.745596    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2748e587-acac-4006-a273-e9ddfdd34657-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.754030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.753947    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-maas-api-tls\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.755595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.755562    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-metrics-certs\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.780052 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.780014    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhvtj\" (UniqueName: \"kubernetes.io/projected/2748e587-acac-4006-a273-e9ddfdd34657-kube-api-access-dhvtj\") pod \"maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.823928 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.823886    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:54.885160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.885044    6011 generic.go:356] "Generic (PLEG): container finished" podID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerID="96835eacd10338c7e7fa395ae015b61338ca5b9c829724f1e97e6f783eab410d" exitCode=0
Oct 30 01:19:54.885506 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:54.885428    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" event={"ID":"11d158e9-6fc5-4b2f-82a7-b5cda7f85187","Type":"ContainerDied","Data":"96835eacd10338c7e7fa395ae015b61338ca5b9c829724f1e97e6f783eab410d"}
Oct 30 01:19:55.045485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.045455    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"]
Oct 30 01:19:55.055021 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.054994    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 01:19:55.162101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.151651    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-kube-api-access-tpmpk\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-kube-api-access-tpmpk\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.162934 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.162901    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-token\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-token\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.163166    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-certs\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.163811    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-data\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.163851    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istiod-ca-cert\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.163984    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-socket\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.164027    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-envoy\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.164100    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-credential-socket\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.164157    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-podinfo\") pod \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\" (UID: \"11d158e9-6fc5-4b2f-82a7-b5cda7f85187\") "
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.166208    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-socket" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.166681    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istiod-ca-cert" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:19:55.178200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.167346    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-credential-socket" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:55.179058 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.178946    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-certs" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:55.206234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.198958    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-envoy" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:55.219667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.207077    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"]
Oct 30 01:19:55.219667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.210421    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-token" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:55.219667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.210568    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-kube-api-access-tpmpk" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "kube-api-access-tpmpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:19:55.219667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.212789    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-podinfo" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:19:55.243048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.241767    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-data" pod "11d158e9-6fc5-4b2f-82a7-b5cda7f85187" (UID: "11d158e9-6fc5-4b2f-82a7-b5cda7f85187"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272668    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpmpk\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-kube-api-access-tpmpk\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272712    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272728    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272739    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272751    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272764    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272779    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272792    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.272798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.272804    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11d158e9-6fc5-4b2f-82a7-b5cda7f85187-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:19:55.581128 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.581091    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"]
Oct 30 01:19:55.581879 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.581860    6011 cpu_manager.go:414] "containerMap: removing container" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy"
Oct 30 01:19:55.581879 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.581879    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy"
Oct 30 01:19:55.582066 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.582054    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" containerName="istio-proxy"
Oct 30 01:19:55.588881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.588858    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.622210 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.622152    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"]
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.677782    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136a0179-cd67-4020-8946-60dda0db9b2f-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.677937    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.678113    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.678190    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.678244    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136a0179-cd67-4020-8946-60dda0db9b2f-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.678281    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.678304    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnlnk\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-kube-api-access-cnlnk\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.678363    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.681289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.678391    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782311    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136a0179-cd67-4020-8946-60dda0db9b2f-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782441    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782573    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782621    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782674    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136a0179-cd67-4020-8946-60dda0db9b2f-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782713    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782736    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnlnk\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-kube-api-access-cnlnk\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782798    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.782830    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.784095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.783393    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.785596 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.785570    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136a0179-cd67-4020-8946-60dda0db9b2f-istiod-ca-cert\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.786105 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.786077    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-data\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.786732 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.786708    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-credential-socket\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.787207 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.787163    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-certs\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.807070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.799224    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136a0179-cd67-4020-8946-60dda0db9b2f-istio-podinfo\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.807070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.801721    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-envoy\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.820785 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.820751    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-istio-token\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.833062 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.832985    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnlnk\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-kube-api-access-cnlnk\") pod \"e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") " pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.895681 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.895640    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4" event={"ID":"11d158e9-6fc5-4b2f-82a7-b5cda7f85187","Type":"ContainerDied","Data":"bfbc17d0a2d8f7dd2ceea5eb855c3f99fd2b776af70c061d5dbc2d33cc2483f5"}
Oct 30 01:19:55.895876 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.895649    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"
Oct 30 01:19:55.895876 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.895751    6011 scope.go:122] "RemoveContainer" containerID="96835eacd10338c7e7fa395ae015b61338ca5b9c829724f1e97e6f783eab410d"
Oct 30 01:19:55.897610 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.897582    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp" event={"ID":"2748e587-acac-4006-a273-e9ddfdd34657","Type":"ContainerStarted","Data":"f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5"}
Oct 30 01:19:55.897722 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.897637    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp" event={"ID":"2748e587-acac-4006-a273-e9ddfdd34657","Type":"ContainerStarted","Data":"f5b63b610d9932f62d4f03ce71128ac4130387deb813479f5ff2a1c31ebf9289"}
Oct 30 01:19:55.897862 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.897836    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:19:55.946936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.946896    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:55.951996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.951970    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"]
Oct 30 01:19:55.955787 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.955761    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-8cf33a-openshift-default-79df9cd7c5-hvks4"]
Oct 30 01:19:55.994784 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:55.994404    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp" podStartSLOduration=1.658484227 podStartE2EDuration="1.99438954s" totalImagesPullingTime="335.905313ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:55.990205552 +0000 UTC m=+2438.092959524" watchObservedRunningTime="2026-10-30 01:19:55.99438954 +0000 UTC m=+2438.097143510"
Oct 30 01:19:56.028908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.028871    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11d158e9-6fc5-4b2f-82a7-b5cda7f85187" path="/var/lib/kubelet/pods/11d158e9-6fc5-4b2f-82a7-b5cda7f85187/volumes"
Oct 30 01:19:56.269645 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:19:56.269609    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136a0179_cd67_4020_8946_60dda0db9b2f.slice/crio-48683abbcafaa51353a3919b7ca50a70404731324b499470422ea5ae1612f744 WatchSource:0}: Error finding container 48683abbcafaa51353a3919b7ca50a70404731324b499470422ea5ae1612f744: Status 404 returned error can't find the container with id 48683abbcafaa51353a3919b7ca50a70404731324b499470422ea5ae1612f744
Oct 30 01:19:56.271494 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.271456    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"]
Oct 30 01:19:56.273048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.273017    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:56.273109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.273083    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:56.273149 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.273116    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:19:56.908612 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.908562    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" event={"ID":"136a0179-cd67-4020-8946-60dda0db9b2f","Type":"ContainerStarted","Data":"2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4"}
Oct 30 01:19:56.909228 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.908632    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" event={"ID":"136a0179-cd67-4020-8946-60dda0db9b2f","Type":"ContainerStarted","Data":"48683abbcafaa51353a3919b7ca50a70404731324b499470422ea5ae1612f744"}
Oct 30 01:19:56.948101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.948065    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:19:56.967493 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:56.967434    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" podStartSLOduration=1.967418772 podStartE2EDuration="1.967418772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:19:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:19:56.964262876 +0000 UTC m=+2439.067016864" watchObservedRunningTime="2026-10-30 01:19:56.967418772 +0000 UTC m=+2439.070172743"
Oct 30 01:19:57.912242 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:19:57.912165    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:19:57.963199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:57.962318    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:19:57.963199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:57.962386    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:19:58.963200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:58.962630    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:19:58.963200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:58.962691    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:19:59.953211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:59.952882    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:19:59.953211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:19:59.952943    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:20:00.952817 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:00.952340    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:20:00.952817 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:00.952419    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:20:01.915245 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:01.915205    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:20:01.965206 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:01.959859    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused" start-of-body=
Oct 30 01:20:01.965206 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:01.959934    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.144:15021/healthz/ready\": dial tcp 10.134.0.144:15021: connect: connection refused"
Oct 30 01:20:02.953288 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:02.953250    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:20:02.953738 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:02.953717    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:20:02.954499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:02.954477    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:20:10.069073 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.068996    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6" podUID="747a8df2-d40a-401f-9d18-9acbc47d74f1" containerName="main" containerID="cri-o://329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2" gracePeriod=45
Oct 30 01:20:10.355658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.355622    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:20:10.450570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.450527    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-model-cache\") pod \"747a8df2-d40a-401f-9d18-9acbc47d74f1\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") "
Oct 30 01:20:10.450760 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.450744    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-home\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-home\") pod \"747a8df2-d40a-401f-9d18-9acbc47d74f1\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") "
Oct 30 01:20:10.450893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.450861    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-model-cache" pod "747a8df2-d40a-401f-9d18-9acbc47d74f1" (UID: "747a8df2-d40a-401f-9d18-9acbc47d74f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:10.450967 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.450906    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/747a8df2-d40a-401f-9d18-9acbc47d74f1-tls-certs\" (UniqueName: \"kubernetes.io/secret/747a8df2-d40a-401f-9d18-9acbc47d74f1-tls-certs\") pod \"747a8df2-d40a-401f-9d18-9acbc47d74f1\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") "
Oct 30 01:20:10.450967 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.450939    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/747a8df2-d40a-401f-9d18-9acbc47d74f1-kube-api-access-pspjs\" (UniqueName: \"kubernetes.io/projected/747a8df2-d40a-401f-9d18-9acbc47d74f1-kube-api-access-pspjs\") pod \"747a8df2-d40a-401f-9d18-9acbc47d74f1\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") "
Oct 30 01:20:10.451078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.450987    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-dshm\") pod \"747a8df2-d40a-401f-9d18-9acbc47d74f1\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") "
Oct 30 01:20:10.451078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.451017    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-tmp-dir\") pod \"747a8df2-d40a-401f-9d18-9acbc47d74f1\" (UID: \"747a8df2-d40a-401f-9d18-9acbc47d74f1\") "
Oct 30 01:20:10.451078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.451030    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-home" pod "747a8df2-d40a-401f-9d18-9acbc47d74f1" (UID: "747a8df2-d40a-401f-9d18-9acbc47d74f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:10.451550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.451523    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-tmp-dir" pod "747a8df2-d40a-401f-9d18-9acbc47d74f1" (UID: "747a8df2-d40a-401f-9d18-9acbc47d74f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:10.451693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.451671    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:20:10.451755 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.451700    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:20:10.451755 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.451714    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:20:10.453450 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.453422    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/747a8df2-d40a-401f-9d18-9acbc47d74f1-kube-api-access-pspjs" pod "747a8df2-d40a-401f-9d18-9acbc47d74f1" (UID: "747a8df2-d40a-401f-9d18-9acbc47d74f1"). InnerVolumeSpecName "kube-api-access-pspjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:20:10.453571 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.453511    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/747a8df2-d40a-401f-9d18-9acbc47d74f1-tls-certs" pod "747a8df2-d40a-401f-9d18-9acbc47d74f1" (UID: "747a8df2-d40a-401f-9d18-9acbc47d74f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:20:10.453571 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.453536    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-dshm" pod "747a8df2-d40a-401f-9d18-9acbc47d74f1" (UID: "747a8df2-d40a-401f-9d18-9acbc47d74f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:20:10.553215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.553158    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/747a8df2-d40a-401f-9d18-9acbc47d74f1-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:20:10.553215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.553214    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pspjs\" (UniqueName: \"kubernetes.io/projected/747a8df2-d40a-401f-9d18-9acbc47d74f1-kube-api-access-pspjs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:20:10.553215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:10.553228    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/747a8df2-d40a-401f-9d18-9acbc47d74f1-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:20:11.058384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.058345    6011 generic.go:356] "Generic (PLEG): container finished" podID="747a8df2-d40a-401f-9d18-9acbc47d74f1" containerID="329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2" exitCode=0
Oct 30 01:20:11.058674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.058404    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"
Oct 30 01:20:11.058674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.058436    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6" event={"ID":"747a8df2-d40a-401f-9d18-9acbc47d74f1","Type":"ContainerDied","Data":"329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2"}
Oct 30 01:20:11.058674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.058489    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6" event={"ID":"747a8df2-d40a-401f-9d18-9acbc47d74f1","Type":"ContainerDied","Data":"66973dc77430e44a815ff315e1726229bec811d9837f92a18e35665710f8e175"}
Oct 30 01:20:11.058674 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.058512    6011 scope.go:122] "RemoveContainer" containerID="329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2"
Oct 30 01:20:11.070873 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.070729    6011 scope.go:122] "RemoveContainer" containerID="329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2"
Oct 30 01:20:11.071143 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:20:11.071025    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2\": container with ID starting with 329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2 not found: ID does not exist" containerID="329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2"
Oct 30 01:20:11.071143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.071051    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2"} err="failed to get container status \"329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2\": rpc error: code = NotFound desc = could not find container \"329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2\": container with ID starting with 329c6c7b6273a812a3a7c337b6db78ae7ef2b492ab2468a4e59ffe7ac63da4d2 not found: ID does not exist"
Oct 30 01:20:11.116290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.116248    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"]
Oct 30 01:20:11.149602 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:11.149556    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w0-283eb6/sub-test-model-283eb6-kserve-5899f749cc-clcl6"]
Oct 30 01:20:12.023965 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.023928    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="747a8df2-d40a-401f-9d18-9acbc47d74f1" path="/var/lib/kubelet/pods/747a8df2-d40a-401f-9d18-9acbc47d74f1/volumes"
Oct 30 01:20:12.645682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.645644    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"]
Oct 30 01:20:12.646470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.646454    6011 cpu_manager.go:414] "containerMap: removing container" podUID="747a8df2-d40a-401f-9d18-9acbc47d74f1" containerName="main"
Oct 30 01:20:12.646519 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.646472    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="747a8df2-d40a-401f-9d18-9acbc47d74f1" containerName="main"
Oct 30 01:20:12.646587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.646577    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="747a8df2-d40a-401f-9d18-9acbc47d74f1" containerName="main"
Oct 30 01:20:12.650340 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.650308    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.673276 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.673236    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"]
Oct 30 01:20:12.682583 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.682548    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-07a3a8-serving-cert\""
Oct 30 01:20:12.795235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.795196    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-metrics-certs\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.795446 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.795258    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-maas-api-tls\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.795446 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.795418    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dpjh\" (UniqueName: \"kubernetes.io/projected/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-kube-api-access-2dpjh\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.795541 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.795464    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.905291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.900931    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-maas-api-tls\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.905291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.901157    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dpjh\" (UniqueName: \"kubernetes.io/projected/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-kube-api-access-2dpjh\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.905291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.901235    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.905291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.901503    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-metrics-certs\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.905291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.903715    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.944286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.925871    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-metrics-certs\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.954250 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.945447    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dpjh\" (UniqueName: \"kubernetes.io/projected/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-kube-api-access-2dpjh\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.955295 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.955263    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-maas-api-tls\") pod \"maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:12.973471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:12.973402    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:13.300013 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:20:13.299981    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c9ce8d_be5c_46b1_8b90_3f8a3b75fc99.slice/crio-2f3544c165b109df2e484f1fe64dbc870a4e172a025e237ea6afaf240d467154 WatchSource:0}: Error finding container 2f3544c165b109df2e484f1fe64dbc870a4e172a025e237ea6afaf240d467154: Status 404 returned error can't find the container with id 2f3544c165b109df2e484f1fe64dbc870a4e172a025e237ea6afaf240d467154
Oct 30 01:20:13.302405 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:13.302379    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"]
Oct 30 01:20:14.092956 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:14.092886    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg" event={"ID":"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99","Type":"ContainerStarted","Data":"f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643"}
Oct 30 01:20:14.092956 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:14.092955    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg" event={"ID":"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99","Type":"ContainerStarted","Data":"2f3544c165b109df2e484f1fe64dbc870a4e172a025e237ea6afaf240d467154"}
Oct 30 01:20:14.093562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:14.093215    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:14.136840 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:14.136788    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg" podStartSLOduration=1.702474069 podStartE2EDuration="2.136774195s" totalImagesPullingTime="434.300126ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:20:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:20:14.134456282 +0000 UTC m=+2456.237210253" watchObservedRunningTime="2026-10-30 01:20:14.136774195 +0000 UTC m=+2456.239528165"
Oct 30 01:20:20.105387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:20.105354    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:20:27.148934 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.148894    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"]
Oct 30 01:20:27.155689 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.155661    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.171985 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.171945    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"]
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.254789    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cda466-cc83-4396-9adf-42cf7db07cfe-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.254909    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxx2g\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-kube-api-access-lxx2g\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.254983    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.255088    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.255148    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.255209    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.255338    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.255374    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.259211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.255430    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.263864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.263832    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-5c515e-openshift-default-dockercfg-cqchw\""
Oct 30 01:20:27.356631 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356596    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cda466-cc83-4396-9adf-42cf7db07cfe-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.356846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356669    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxx2g\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-kube-api-access-lxx2g\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.356846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356713    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.356846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356777    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.356846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356808    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.356846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356834    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356950    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.356977    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.357022    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357398 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.357266    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357398 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.357345    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357515 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.357438    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357573 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.357538    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.357573 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.357558    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cda466-cc83-4396-9adf-42cf7db07cfe-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.359597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.359576    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.359714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.359697    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.378462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.378429    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.378660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.378638    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxx2g\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-kube-api-access-lxx2g\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.479591 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.479498    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:27.801312 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:20:27.801276    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40cda466_cc83_4396_9adf_42cf7db07cfe.slice/crio-9f728e5dfc3ba16b69ec6219f8dbcd760a94d1b67d34b00ea9f6b560c667bddb WatchSource:0}: Error finding container 9f728e5dfc3ba16b69ec6219f8dbcd760a94d1b67d34b00ea9f6b560c667bddb: Status 404 returned error can't find the container with id 9f728e5dfc3ba16b69ec6219f8dbcd760a94d1b67d34b00ea9f6b560c667bddb
Oct 30 01:20:27.802305 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.802279    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"]
Oct 30 01:20:27.804246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.804209    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:20:27.804343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.804300    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:20:27.804343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:27.804331    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:20:27.913255 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:20:27.913167    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:20:28.248994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:28.248904    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" event={"ID":"40cda466-cc83-4396-9adf-42cf7db07cfe","Type":"ContainerStarted","Data":"38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25"}
Oct 30 01:20:28.248994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:28.248973    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" event={"ID":"40cda466-cc83-4396-9adf-42cf7db07cfe","Type":"ContainerStarted","Data":"9f728e5dfc3ba16b69ec6219f8dbcd760a94d1b67d34b00ea9f6b560c667bddb"}
Oct 30 01:20:28.321668 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:28.321608    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podStartSLOduration=1.321591185 podStartE2EDuration="1.321591185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:20:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:20:28.305485709 +0000 UTC m=+2470.408239679" watchObservedRunningTime="2026-10-30 01:20:28.321591185 +0000 UTC m=+2470.424345228"
Oct 30 01:20:28.480651 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:28.480609    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:28.482237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:28.482207    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 01:20:28.482361 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:28.482275    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 01:20:29.484515 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:29.483834    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 01:20:29.484515 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:29.483882    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 01:20:30.480806 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:30.480766    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 01:20:30.481019 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:30.480827    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 01:20:31.480349 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:31.480313    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 01:20:31.480759 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:31.480377    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 01:20:32.481986 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:32.481529    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 01:20:32.481986 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:32.481596    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 01:20:33.480522 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:33.480484    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused" start-of-body=
Oct 30 01:20:33.480718 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:33.480549    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.146:15021/healthz/ready\": dial tcp 10.134.0.146:15021: connect: connection refused"
Oct 30 01:20:34.485807 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:34.485776    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:34.486291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:34.486271    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:34.486989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:34.486968    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:20:47.760953 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:47.760915    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"]
Oct 30 01:20:55.192151 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.192104    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"]
Oct 30 01:20:55.197324 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.197298    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.214910 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.214880    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"]
Oct 30 01:20:55.224280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.224248    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-ad00a0-openshift-default-dockercfg-w2ff8\""
Oct 30 01:20:55.336583 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.336541    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-token\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.336806 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.336624    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-certs\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.336806 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.336766    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-envoy\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.336930 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.336826    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwzfz\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-kube-api-access-hwzfz\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.336980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.336949    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c781a89-e008-4ae5-a88c-8f022004bcc8-istiod-ca-cert\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.337030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.336983    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-data\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.337030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.337002    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-socket\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.337030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.337019    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-podinfo\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.337141 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.337107    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-credential-socket\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439114    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-token\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439255    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-certs\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439328    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-envoy\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439369    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwzfz\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-kube-api-access-hwzfz\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439558 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439482    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c781a89-e008-4ae5-a88c-8f022004bcc8-istiod-ca-cert\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439558 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439520    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-data\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439558 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439550    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-socket\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439730 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439578    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-podinfo\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439730 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439628    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-credential-socket\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439851 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439771    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-certs\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.439989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439940    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-data\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.440069 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.439992    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-socket\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.440130 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.440114    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-credential-socket\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.440253 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.440232    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c781a89-e008-4ae5-a88c-8f022004bcc8-istiod-ca-cert\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.442320 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.442260    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-envoy\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.442320 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.442308    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-podinfo\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.459505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.459477    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-token\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.459654 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.459529    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwzfz\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-kube-api-access-hwzfz\") pod \"e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") " pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.515597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.515565    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:55.835261 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:20:55.835227    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c781a89_e008_4ae5_a88c_8f022004bcc8.slice/crio-bb2f9e2f8a880fed29556cea258db87b60ec79e24b7946e6d574a5945f161ba7 WatchSource:0}: Error finding container bb2f9e2f8a880fed29556cea258db87b60ec79e24b7946e6d574a5945f161ba7: Status 404 returned error can't find the container with id bb2f9e2f8a880fed29556cea258db87b60ec79e24b7946e6d574a5945f161ba7
Oct 30 01:20:55.835919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.835893    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"]
Oct 30 01:20:55.838424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.838390    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:20:55.838516 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.838457    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:20:55.838516 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:55.838489    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:20:56.524132 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:56.524063    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" event={"ID":"0c781a89-e008-4ae5-a88c-8f022004bcc8","Type":"ContainerStarted","Data":"47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5"}
Oct 30 01:20:56.524132 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:56.524129    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" event={"ID":"0c781a89-e008-4ae5-a88c-8f022004bcc8","Type":"ContainerStarted","Data":"bb2f9e2f8a880fed29556cea258db87b60ec79e24b7946e6d574a5945f161ba7"}
Oct 30 01:20:56.582671 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:56.582616    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podStartSLOduration=1.582603641 podStartE2EDuration="1.582603641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:20:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:20:56.580540553 +0000 UTC m=+2498.683294523" watchObservedRunningTime="2026-10-30 01:20:56.582603641 +0000 UTC m=+2498.685357611"
Oct 30 01:20:57.516229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:57.516151    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:20:57.518186 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:57.518144    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 01:20:57.518327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:57.518209    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 01:20:57.913340 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:20:57.913298    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:20:58.516214 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:58.516157    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 01:20:58.516397 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:58.516238    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 01:20:59.516223 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:59.516162    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 01:20:59.516769 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:20:59.516249    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 01:21:00.516495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:00.516457    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 01:21:00.517034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:00.516524    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 01:21:01.516749 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:01.516712    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 01:21:01.517281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:01.516772    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 01:21:02.516862 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:02.516812    6011 patch_prober.go:29] interesting pod/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused" start-of-body=
Oct 30 01:21:02.517327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:02.516898    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.147:15021/healthz/ready\": dial tcp 10.134.0.147:15021: connect: connection refused"
Oct 30 01:21:02.783999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:02.783959    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl" podUID="465bd602-695a-45e6-b8ab-0aac9f2b2846" containerName="main" containerID="cri-o://76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073" gracePeriod=45
Oct 30 01:21:03.079866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.079836    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:21:03.129202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.111082    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/465bd602-695a-45e6-b8ab-0aac9f2b2846-tls-certs\" (UniqueName: \"kubernetes.io/secret/465bd602-695a-45e6-b8ab-0aac9f2b2846-tls-certs\") pod \"465bd602-695a-45e6-b8ab-0aac9f2b2846\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") "
Oct 30 01:21:03.129202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.111279    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-model-cache\") pod \"465bd602-695a-45e6-b8ab-0aac9f2b2846\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") "
Oct 30 01:21:03.129202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.111341    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-tmp-dir\") pod \"465bd602-695a-45e6-b8ab-0aac9f2b2846\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") "
Oct 30 01:21:03.129202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.111495    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-home\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-home\") pod \"465bd602-695a-45e6-b8ab-0aac9f2b2846\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") "
Oct 30 01:21:03.129202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.111587    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/465bd602-695a-45e6-b8ab-0aac9f2b2846-kube-api-access-x4kg9\" (UniqueName: \"kubernetes.io/projected/465bd602-695a-45e6-b8ab-0aac9f2b2846-kube-api-access-x4kg9\") pod \"465bd602-695a-45e6-b8ab-0aac9f2b2846\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") "
Oct 30 01:21:03.129202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.111626    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-dshm\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-dshm\") pod \"465bd602-695a-45e6-b8ab-0aac9f2b2846\" (UID: \"465bd602-695a-45e6-b8ab-0aac9f2b2846\") "
Oct 30 01:21:03.129202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.124625    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-dshm" pod "465bd602-695a-45e6-b8ab-0aac9f2b2846" (UID: "465bd602-695a-45e6-b8ab-0aac9f2b2846"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:03.145993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.138270    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-tmp-dir" pod "465bd602-695a-45e6-b8ab-0aac9f2b2846" (UID: "465bd602-695a-45e6-b8ab-0aac9f2b2846"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:03.146355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.146310    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-home" pod "465bd602-695a-45e6-b8ab-0aac9f2b2846" (UID: "465bd602-695a-45e6-b8ab-0aac9f2b2846"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:03.170268 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.164481    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-model-cache" pod "465bd602-695a-45e6-b8ab-0aac9f2b2846" (UID: "465bd602-695a-45e6-b8ab-0aac9f2b2846"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:03.206129 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.206067    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/465bd602-695a-45e6-b8ab-0aac9f2b2846-kube-api-access-x4kg9" pod "465bd602-695a-45e6-b8ab-0aac9f2b2846" (UID: "465bd602-695a-45e6-b8ab-0aac9f2b2846"). InnerVolumeSpecName "kube-api-access-x4kg9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:03.214015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.213717    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:03.214015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.213749    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:03.214015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.213757    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:03.214015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.213766    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4kg9\" (UniqueName: \"kubernetes.io/projected/465bd602-695a-45e6-b8ab-0aac9f2b2846-kube-api-access-x4kg9\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:03.214015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.213775    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/465bd602-695a-45e6-b8ab-0aac9f2b2846-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:03.216136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.215940    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/465bd602-695a-45e6-b8ab-0aac9f2b2846-tls-certs" pod "465bd602-695a-45e6-b8ab-0aac9f2b2846" (UID: "465bd602-695a-45e6-b8ab-0aac9f2b2846"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:21:03.315752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.315664    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/465bd602-695a-45e6-b8ab-0aac9f2b2846-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:03.539216 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.539129    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:21:03.541224 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.541189    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:21:03.543061 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.543041    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:21:03.769247 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.769118    6011 generic.go:356] "Generic (PLEG): container finished" podID="465bd602-695a-45e6-b8ab-0aac9f2b2846" containerID="76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073" exitCode=0
Oct 30 01:21:03.769611 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.769583    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl" event={"ID":"465bd602-695a-45e6-b8ab-0aac9f2b2846","Type":"ContainerDied","Data":"76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073"}
Oct 30 01:21:03.769714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.769631    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl" event={"ID":"465bd602-695a-45e6-b8ab-0aac9f2b2846","Type":"ContainerDied","Data":"e2c60e0c169e361b1f432a4b228d7501e9c62113385cfad3e44dcfce1a0201e6"}
Oct 30 01:21:03.769714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.769649    6011 scope.go:122] "RemoveContainer" containerID="76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073"
Oct 30 01:21:03.770804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.770778    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"
Oct 30 01:21:03.800958 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.800767    6011 scope.go:122] "RemoveContainer" containerID="76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073"
Oct 30 01:21:03.801380 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:21:03.801346    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073\": container with ID starting with 76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073 not found: ID does not exist" containerID="76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073"
Oct 30 01:21:03.801528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.801392    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073"} err="failed to get container status \"76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073\": rpc error: code = NotFound desc = could not find container \"76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073\": container with ID starting with 76d139fd9a7fc9aa493e9a3a8b7b05fce85e7d019712d81b48a11375999db073 not found: ID does not exist"
Oct 30 01:21:03.845056 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.845022    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"]
Oct 30 01:21:03.851371 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:03.851338    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w1-cb31e3/auth-test-model-cb31e3-kserve-74cd84b5d5-7wfbl"]
Oct 30 01:21:04.025827 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:04.025735    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="465bd602-695a-45e6-b8ab-0aac9f2b2846" path="/var/lib/kubelet/pods/465bd602-695a-45e6-b8ab-0aac9f2b2846/volumes"
Oct 30 01:21:08.419720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.419687    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"]
Oct 30 01:21:08.420210 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.419933    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss" podUID="35903ffc-005b-4c93-a491-15fb07a32c2e" containerName="maas-api" containerID="cri-o://78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140" gracePeriod=30
Oct 30 01:21:08.765219 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.764560    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:21:08.811200 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.809204    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/35903ffc-005b-4c93-a491-15fb07a32c2e-kube-api-access-gprvz\" (UniqueName: \"kubernetes.io/projected/35903ffc-005b-4c93-a491-15fb07a32c2e-kube-api-access-gprvz\") pod \"35903ffc-005b-4c93-a491-15fb07a32c2e\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") "
Oct 30 01:21:08.817283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.811669    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-metrics-certs\") pod \"35903ffc-005b-4c93-a491-15fb07a32c2e\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") "
Oct 30 01:21:08.817283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.812547    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-maas-api-tls\") pod \"35903ffc-005b-4c93-a491-15fb07a32c2e\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") "
Oct 30 01:21:08.817283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.815573    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/35903ffc-005b-4c93-a491-15fb07a32c2e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35903ffc-005b-4c93-a491-15fb07a32c2e-openshift-service-ca\") pod \"35903ffc-005b-4c93-a491-15fb07a32c2e\" (UID: \"35903ffc-005b-4c93-a491-15fb07a32c2e\") "
Oct 30 01:21:08.817283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.817124    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35903ffc-005b-4c93-a491-15fb07a32c2e-openshift-service-ca" pod "35903ffc-005b-4c93-a491-15fb07a32c2e" (UID: "35903ffc-005b-4c93-a491-15fb07a32c2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:21:08.875635 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.875583    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-metrics-certs" pod "35903ffc-005b-4c93-a491-15fb07a32c2e" (UID: "35903ffc-005b-4c93-a491-15fb07a32c2e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:21:08.912608 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.906984    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35903ffc-005b-4c93-a491-15fb07a32c2e-kube-api-access-gprvz" pod "35903ffc-005b-4c93-a491-15fb07a32c2e" (UID: "35903ffc-005b-4c93-a491-15fb07a32c2e"). InnerVolumeSpecName "kube-api-access-gprvz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:08.912608 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.907450    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-maas-api-tls" pod "35903ffc-005b-4c93-a491-15fb07a32c2e" (UID: "35903ffc-005b-4c93-a491-15fb07a32c2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:21:08.920498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.919549    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35903ffc-005b-4c93-a491-15fb07a32c2e-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:08.920498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.919592    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gprvz\" (UniqueName: \"kubernetes.io/projected/35903ffc-005b-4c93-a491-15fb07a32c2e-kube-api-access-gprvz\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:08.920498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.919607    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:08.920498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.919623    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35903ffc-005b-4c93-a491-15fb07a32c2e-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:08.971204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.970213    6011 generic.go:356] "Generic (PLEG): container finished" podID="35903ffc-005b-4c93-a491-15fb07a32c2e" containerID="78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140" exitCode=0
Oct 30 01:21:08.971204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.970257    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss" event={"ID":"35903ffc-005b-4c93-a491-15fb07a32c2e","Type":"ContainerDied","Data":"78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140"}
Oct 30 01:21:08.971204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.970310    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"
Oct 30 01:21:08.971204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.970323    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss" event={"ID":"35903ffc-005b-4c93-a491-15fb07a32c2e","Type":"ContainerDied","Data":"e829d2401b3210afc066b9f34490dd89ba97dc4bc5c2b452adf07740802516dc"}
Oct 30 01:21:08.971204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.970355    6011 scope.go:122] "RemoveContainer" containerID="78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140"
Oct 30 01:21:08.991387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.991356    6011 scope.go:122] "RemoveContainer" containerID="78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140"
Oct 30 01:21:08.991800 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:21:08.991776    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140\": container with ID starting with 78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140 not found: ID does not exist" containerID="78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140"
Oct 30 01:21:08.991891 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:08.991811    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140"} err="failed to get container status \"78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140\": rpc error: code = NotFound desc = could not find container \"78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140\": container with ID starting with 78b5f0efb067d576dc8fe348389d144c437895dcb9813060a2ced9095d853140 not found: ID does not exist"
Oct 30 01:21:09.027697 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:09.027659    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"]
Oct 30 01:21:09.031850 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:09.031824    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w0-283eb6-7bb5dd86d7-4dfss"]
Oct 30 01:21:10.023814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:10.023774    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="35903ffc-005b-4c93-a491-15fb07a32c2e" path="/var/lib/kubelet/pods/35903ffc-005b-4c93-a491-15fb07a32c2e/volumes"
Oct 30 01:21:11.419401 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.419361    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"]
Oct 30 01:21:11.427028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.420686    6011 cpu_manager.go:414] "containerMap: removing container" podUID="35903ffc-005b-4c93-a491-15fb07a32c2e" containerName="maas-api"
Oct 30 01:21:11.427028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.420720    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="35903ffc-005b-4c93-a491-15fb07a32c2e" containerName="maas-api"
Oct 30 01:21:11.427028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.420750    6011 cpu_manager.go:414] "containerMap: removing container" podUID="465bd602-695a-45e6-b8ab-0aac9f2b2846" containerName="main"
Oct 30 01:21:11.427028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.420760    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="465bd602-695a-45e6-b8ab-0aac9f2b2846" containerName="main"
Oct 30 01:21:11.427028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.420991    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="35903ffc-005b-4c93-a491-15fb07a32c2e" containerName="maas-api"
Oct 30 01:21:11.427028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.421006    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="465bd602-695a-45e6-b8ab-0aac9f2b2846" containerName="main"
Oct 30 01:21:11.436367 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.433304    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.437917 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.437885    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"]
Oct 30 01:21:11.479666 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.479631    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-07a3a8-openshift-default-dockercfg-xw9br\""
Oct 30 01:21:11.599924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.599882    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b92022e8-a46c-45cd-a161-cad0a4af03f0-istiod-ca-cert\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600049    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-envoy\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600089    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-credential-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600110    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-data\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600315 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600292    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600428 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600411    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-token\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600497 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600478    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxbl\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-kube-api-access-9dxbl\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600542 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600534    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-podinfo\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.600626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.600603    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-certs\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.702721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.702605    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.702721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.702714    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-token\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.702970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.702778    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxbl\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-kube-api-access-9dxbl\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.702970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.702859    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-podinfo\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.702970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.702902    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-certs\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.703144 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.702975    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b92022e8-a46c-45cd-a161-cad0a4af03f0-istiod-ca-cert\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.703144 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.703112    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-envoy\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.703292 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.703156    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-credential-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.703292 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.703202    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.703292 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.703208    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-data\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.703573 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.703550    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-data\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.703647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.703615    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-certs\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.704389 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.704358    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b92022e8-a46c-45cd-a161-cad0a4af03f0-istiod-ca-cert\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.704868 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.704785    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-credential-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.707935 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.707121    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-envoy\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.707935 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.707456    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-podinfo\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.726138 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.726108    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxbl\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-kube-api-access-9dxbl\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.726333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.726147    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-token\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:11.760159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:11.760113    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:12.086701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.086667    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"]
Oct 30 01:21:12.086845 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:21:12.086669    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb92022e8_a46c_45cd_a161_cad0a4af03f0.slice/crio-f577e335e998760f8e4cb951c43e8c2ab191b63db9ad443073e713da3038603c WatchSource:0}: Error finding container f577e335e998760f8e4cb951c43e8c2ab191b63db9ad443073e713da3038603c: Status 404 returned error can't find the container with id f577e335e998760f8e4cb951c43e8c2ab191b63db9ad443073e713da3038603c
Oct 30 01:21:12.090972 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.090925    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:12.091092 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.091024    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:12.091092 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.091073    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:12.255698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.255660    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"]
Oct 30 01:21:12.262243 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.262209    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.280438 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.280390    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"]
Oct 30 01:21:12.425400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.424534    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51759780-e51e-4310-be1b-f82b6c56f28d-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.426080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.426053    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.426428 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.426398    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.426627 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.426608    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51759780-e51e-4310-be1b-f82b6c56f28d-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.426819 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.426803    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s94gq\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-kube-api-access-s94gq\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.427012 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.426996    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.427324 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.427233    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.427513 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.427496    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.427771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.427721    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529077 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529044    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51759780-e51e-4310-be1b-f82b6c56f28d-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529090    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529119    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529149    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51759780-e51e-4310-be1b-f82b6c56f28d-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529193    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s94gq\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-kube-api-access-s94gq\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529243    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529292    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529313    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529717 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529370    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529703    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529871    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.529942 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.529914    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51759780-e51e-4310-be1b-f82b6c56f28d-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.530163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.530132    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.530326 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.530303    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.532238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.532211    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.532352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.532336    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51759780-e51e-4310-be1b-f82b6c56f28d-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.549713 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.549683    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s94gq\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-kube-api-access-s94gq\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.549865 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.549740    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.596957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.596921    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:12.922006 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:21:12.921967    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51759780_e51e_4310_be1b_f82b6c56f28d.slice/crio-3d0e3d2dda79ab8a69662dae4c3b47531cc05bed695b832641c29e64e31716da WatchSource:0}: Error finding container 3d0e3d2dda79ab8a69662dae4c3b47531cc05bed695b832641c29e64e31716da: Status 404 returned error can't find the container with id 3d0e3d2dda79ab8a69662dae4c3b47531cc05bed695b832641c29e64e31716da
Oct 30 01:21:12.922865 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.922843    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"]
Oct 30 01:21:12.925545 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.925502    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:12.925645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.925589    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:12.925645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:12.925635    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:13.005082 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:13.005041    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" event={"ID":"b92022e8-a46c-45cd-a161-cad0a4af03f0","Type":"ContainerStarted","Data":"c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436"}
Oct 30 01:21:13.005274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:13.005114    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" event={"ID":"b92022e8-a46c-45cd-a161-cad0a4af03f0","Type":"ContainerStarted","Data":"f577e335e998760f8e4cb951c43e8c2ab191b63db9ad443073e713da3038603c"}
Oct 30 01:21:13.006718 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:13.006689    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" event={"ID":"51759780-e51e-4310-be1b-f82b6c56f28d","Type":"ContainerStarted","Data":"3d0e3d2dda79ab8a69662dae4c3b47531cc05bed695b832641c29e64e31716da"}
Oct 30 01:21:13.765616 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:13.764572    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:13.771384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:13.768315    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Oct 30 01:21:13.771384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:13.768396    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Oct 30 01:21:14.022638 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:14.022553    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" event={"ID":"51759780-e51e-4310-be1b-f82b6c56f28d","Type":"ContainerStarted","Data":"e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a"}
Oct 30 01:21:14.071197 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:14.070424    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" podStartSLOduration=3.070398772 podStartE2EDuration="3.070398772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:13.062738389 +0000 UTC m=+2515.165492360" watchObservedRunningTime="2026-10-30 01:21:14.070398772 +0000 UTC m=+2516.173152743"
Oct 30 01:21:14.597676 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:14.597631    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:14.599263 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:14.599238    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:21:14.599362 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:14.599296    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:21:14.768364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:14.768319    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Oct 30 01:21:14.770060 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:14.770015    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Oct 30 01:21:15.606205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:15.605493    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:21:15.606205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:15.605561    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:21:15.762211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:15.761610    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Oct 30 01:21:15.762211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:15.761679    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Oct 30 01:21:16.597661 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:16.597625    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:21:16.598167 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:16.597676    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:21:16.770208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:16.762951    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Oct 30 01:21:16.770208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:16.763013    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Oct 30 01:21:16.797204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:16.773561    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podStartSLOduration=4.773539156 podStartE2EDuration="4.773539156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:14.072359331 +0000 UTC m=+2516.175113324" watchObservedRunningTime="2026-10-30 01:21:16.773539156 +0000 UTC m=+2518.876293128"
Oct 30 01:21:16.811455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:16.811421    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"]
Oct 30 01:21:16.811890 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:16.811854    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy" containerID="cri-o://270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f" gracePeriod=30
Oct 30 01:21:17.598274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:17.597659    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:21:17.598274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:17.597720    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:21:17.770199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:17.763302    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused" start-of-body=
Oct 30 01:21:17.770199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:17.763370    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.148:15021/healthz/ready\": dial tcp 10.134.0.148:15021: connect: connection refused"
Oct 30 01:21:18.597858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:18.597819    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:21:18.598043 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:18.597882    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:21:18.614649 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:18.614610    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:21:18.615042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:18.614675    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:21:18.804918 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:18.804879    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:18.805502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:18.805476    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:18.806753 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:18.806729    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:21:19.623196 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:19.615299    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused" start-of-body=
Oct 30 01:21:19.650876 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:19.650840    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.149:15021/healthz/ready\": dial tcp 10.134.0.149:15021: connect: connection refused"
Oct 30 01:21:20.651531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:20.651498    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:20.652998 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:20.652977    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:20.653749 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:20.653730    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:21:22.125125 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.125086    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:21:22.169686 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.169650    6011 generic.go:356] "Generic (PLEG): container finished" podID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerID="270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f" exitCode=0
Oct 30 01:21:22.169863 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.169703    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"
Oct 30 01:21:22.169863 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.169745    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" event={"ID":"daaae85b-371c-4f62-8d80-dfaa7cb7688e","Type":"ContainerDied","Data":"270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f"}
Oct 30 01:21:22.169863 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.169836    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz" event={"ID":"daaae85b-371c-4f62-8d80-dfaa7cb7688e","Type":"ContainerDied","Data":"f94099816a74892ca11e3d4d08da20da392f8c8043d4ef7732c88232dcbae363"}
Oct 30 01:21:22.170051 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.169866    6011 scope.go:122] "RemoveContainer" containerID="270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f"
Oct 30 01:21:22.181807 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.181786    6011 scope.go:122] "RemoveContainer" containerID="270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f"
Oct 30 01:21:22.182230 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:21:22.182199    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f\": container with ID starting with 270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f not found: ID does not exist" containerID="270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f"
Oct 30 01:21:22.182429 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.182393    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f"} err="failed to get container status \"270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f\": rpc error: code = NotFound desc = could not find container \"270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f\": container with ID starting with 270c26a6bd7a06b947447ccd3d39ac68383c5eb76b8b6a69d8cfd2778772361f not found: ID does not exist"
Oct 30 01:21:22.247216 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247141    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-certs\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247226    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-data\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247264    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-kube-api-access-tkbwp\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-kube-api-access-tkbwp\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247289    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-podinfo\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247389    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-token\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-token\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247470    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-certs" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:22.247563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247536    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-credential-socket\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247541    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-data" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:22.247703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247577    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istiod-ca-cert\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247683    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-envoy\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247813    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-socket\") pod \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\" (UID: \"daaae85b-371c-4f62-8d80-dfaa7cb7688e\") "
Oct 30 01:21:22.247834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247818    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-credential-socket" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:22.247974 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.247952    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istiod-ca-cert" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:21:22.248382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.248280    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-socket" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:22.248731 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.248710    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.248824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.248737    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.248824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.248752    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.248824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.248766    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.248824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.248779    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.249945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.249916    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-kube-api-access-tkbwp" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "kube-api-access-tkbwp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:22.249945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.249937    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-token" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:21:22.250071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.249958    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-podinfo" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:21:22.250409 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.250390    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-envoy" pod "daaae85b-371c-4f62-8d80-dfaa7cb7688e" (UID: "daaae85b-371c-4f62-8d80-dfaa7cb7688e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:21:22.350594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.350496    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tkbwp\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-kube-api-access-tkbwp\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.350594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.350531    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.350594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.350541    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.350594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.350549    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daaae85b-371c-4f62-8d80-dfaa7cb7688e-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:21:22.523163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.523115    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"]
Oct 30 01:21:22.538579 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:22.538541    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w0-283eb6-openshift-default-bc7b7cc95-lg6fz"]
Oct 30 01:21:24.024363 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:24.024322    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" path="/var/lib/kubelet/pods/daaae85b-371c-4f62-8d80-dfaa7cb7688e/volumes"
Oct 30 01:21:27.283699 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.283657    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"]
Oct 30 01:21:27.284502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.284485    6011 cpu_manager.go:414] "containerMap: removing container" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy"
Oct 30 01:21:27.284502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.284502    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy"
Oct 30 01:21:27.284618 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.284608    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="daaae85b-371c-4f62-8d80-dfaa7cb7688e" containerName="istio-proxy"
Oct 30 01:21:27.290272 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.290246    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.309723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.309681    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"]
Oct 30 01:21:27.443959 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.443864    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.443959 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.443936    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.444212 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.444003    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.444212 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.444097    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.444212 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.444139    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.444317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.444271    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91adfc76-ab32-46cd-952c-97daab8d00db-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.444355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.444327    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p5bb\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-kube-api-access-2p5bb\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.444441 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.444419    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91adfc76-ab32-46cd-952c-97daab8d00db-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.444515 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.444502    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545526 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545439    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545592    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545612    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545658    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545695    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545734    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91adfc76-ab32-46cd-952c-97daab8d00db-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545820 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545803    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p5bb\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-kube-api-access-2p5bb\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545869    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91adfc76-ab32-46cd-952c-97daab8d00db-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.545980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.545942    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.546089 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.546042    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-data\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.546152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.546104    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-certs\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.546276 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.546227    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-credential-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.546643 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.546612    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-socket\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.546743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.546668    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91adfc76-ab32-46cd-952c-97daab8d00db-istiod-ca-cert\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.548119 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.548099    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-envoy\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.548485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.548464    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91adfc76-ab32-46cd-952c-97daab8d00db-istio-podinfo\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.566333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.566294    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-istio-token\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.566516 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.566412    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p5bb\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-kube-api-access-2p5bb\") pod \"e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") " pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.606507 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.606460    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:27.913304 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:21:27.913222    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:21:27.926593 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:21:27.926548    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91adfc76_ab32_46cd_952c_97daab8d00db.slice/crio-67486d9ac0a0b5739bb02372a54343c081df6ffb64a5a039cc743f77a954b21c WatchSource:0}: Error finding container 67486d9ac0a0b5739bb02372a54343c081df6ffb64a5a039cc743f77a954b21c: Status 404 returned error can't find the container with id 67486d9ac0a0b5739bb02372a54343c081df6ffb64a5a039cc743f77a954b21c
Oct 30 01:21:27.927421 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.927398    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"]
Oct 30 01:21:27.929791 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.929751    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:27.929873 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.929843    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:27.929929 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:27.929894    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:28.223617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:28.223525    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" event={"ID":"91adfc76-ab32-46cd-952c-97daab8d00db","Type":"ContainerStarted","Data":"ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03"}
Oct 30 01:21:28.223617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:28.223588    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" event={"ID":"91adfc76-ab32-46cd-952c-97daab8d00db","Type":"ContainerStarted","Data":"67486d9ac0a0b5739bb02372a54343c081df6ffb64a5a039cc743f77a954b21c"}
Oct 30 01:21:28.275344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:28.275277    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podStartSLOduration=1.275257694 podStartE2EDuration="1.275257694s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:28.269887062 +0000 UTC m=+2530.372641037" watchObservedRunningTime="2026-10-30 01:21:28.275257694 +0000 UTC m=+2530.378011668"
Oct 30 01:21:28.607090 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:28.607047    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:29.608776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:29.608741    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 01:21:29.609266 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:29.608796    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 01:21:30.609770 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:30.609705    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 01:21:30.610439 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:30.610399    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 01:21:31.607079 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:31.607036    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 01:21:31.607329 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:31.607110    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 01:21:32.607508 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:32.607465    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 01:21:32.607960 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:32.607522    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 01:21:33.607495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:33.607454    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 01:21:33.607696 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:33.607528    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 01:21:34.607024 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:34.606971    6011 patch_prober.go:29] interesting pod/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused" start-of-body=
Oct 30 01:21:34.607267 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:34.607038    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.150:15021/healthz/ready\": dial tcp 10.134.0.150:15021: connect: connection refused"
Oct 30 01:21:35.612675 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:35.612637    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:35.613155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:35.613136    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:35.614004 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:35.613983    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:21:41.450916 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.450881    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"]
Oct 30 01:21:41.455655 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.455631    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.481597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.481563    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"]
Oct 30 01:21:41.616968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.616920    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-certs\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617224 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.616976    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-data\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617224 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.617111    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vws6l\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-kube-api-access-vws6l\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617388 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.617276    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istiod-ca-cert\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617388 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.617340    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-envoy\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617474 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.617452    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617508 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.617489    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-token\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617555 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.617514    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-podinfo\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.617591 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.617566    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-credential-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.718726 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.718635    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-token\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.718726 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.718701    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-podinfo\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.718974 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.718731    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-credential-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.718978    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-certs\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719024    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-data\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719115 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719094    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vws6l\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-kube-api-access-vws6l\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719243 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719164    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istiod-ca-cert\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719243 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719239    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-envoy\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719376 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719307    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719376 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719328    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-credential-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.719488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719451    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-certs\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.720300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.719954    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-socket\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.720300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.720240    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-data\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.720300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.720290    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istiod-ca-cert\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.723219 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.721851    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-envoy\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.723219 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.722060    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-podinfo\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.742530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.742463    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vws6l\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-kube-api-access-vws6l\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.742530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.742521    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-token\") pod \"e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") " pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:41.792393 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:41.792353    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:42.125262 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.125227    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"]
Oct 30 01:21:42.126143 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:21:42.126116    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6edb9b4d_ba8c_4679_8ee4_a36537ac205e.slice/crio-67c1f401504f24104ef5a7ad2ed2fb8377425491917d159d25d3b671f36bf735 WatchSource:0}: Error finding container 67c1f401504f24104ef5a7ad2ed2fb8377425491917d159d25d3b671f36bf735: Status 404 returned error can't find the container with id 67c1f401504f24104ef5a7ad2ed2fb8377425491917d159d25d3b671f36bf735
Oct 30 01:21:42.128914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.128879    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:42.128999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.128948    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:42.128999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.128979    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:21:42.390698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.390597    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" event={"ID":"6edb9b4d-ba8c-4679-8ee4-a36537ac205e","Type":"ContainerStarted","Data":"153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3"}
Oct 30 01:21:42.390698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.390660    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" event={"ID":"6edb9b4d-ba8c-4679-8ee4-a36537ac205e","Type":"ContainerStarted","Data":"67c1f401504f24104ef5a7ad2ed2fb8377425491917d159d25d3b671f36bf735"}
Oct 30 01:21:42.437655 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.437600    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" podStartSLOduration=1.437584492 podStartE2EDuration="1.437584492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:42.433581253 +0000 UTC m=+2544.536335227" watchObservedRunningTime="2026-10-30 01:21:42.437584492 +0000 UTC m=+2544.540338518"
Oct 30 01:21:42.794355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:42.794153    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:43.794839 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:43.794795    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 01:21:43.795289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:43.794873    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 01:21:44.793198 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:44.793138    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 01:21:44.793403 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:44.793277    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 01:21:45.793731 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:45.793693    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 01:21:45.794148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:45.793755    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 01:21:46.793500 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:46.793467    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 01:21:46.793684 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:46.793519    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 01:21:47.793118 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:47.793082    6011 patch_prober.go:29] interesting pod/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused" start-of-body=
Oct 30 01:21:47.793549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:47.793149    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.151:15021/healthz/ready\": dial tcp 10.134.0.151:15021: connect: connection refused"
Oct 30 01:21:48.798687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:48.798650    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:48.799161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:48.799149    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:48.800049 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:48.800026    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:21:57.707471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.707434    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"]
Oct 30 01:21:57.713380 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.713355    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.729873 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.729839    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"]
Oct 30 01:21:57.737961 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.737927    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-11fbe8-serving-cert\""
Oct 30 01:21:57.850647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.850609    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-metrics-certs\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.850839 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.850738    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-maas-api-tls\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.850839 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.850808    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9dpt\" (UniqueName: \"kubernetes.io/projected/c094f8f4-a12e-489e-92b3-908016822212-kube-api-access-r9dpt\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.850924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.850846    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094f8f4-a12e-489e-92b3-908016822212-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.936146 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:21:57.936110    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:21:57.952573 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.952537    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-metrics-certs\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.952772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.952620    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-maas-api-tls\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.952772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.952663    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9dpt\" (UniqueName: \"kubernetes.io/projected/c094f8f4-a12e-489e-92b3-908016822212-kube-api-access-r9dpt\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.952772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.952693    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094f8f4-a12e-489e-92b3-908016822212-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.953433 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.953405    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094f8f4-a12e-489e-92b3-908016822212-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.955602 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.955577    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-maas-api-tls\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.955731 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.955603    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-metrics-certs\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:57.975111 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:57.975026    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9dpt\" (UniqueName: \"kubernetes.io/projected/c094f8f4-a12e-489e-92b3-908016822212-kube-api-access-r9dpt\") pod \"maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:58.032070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:58.032034    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:58.351788 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:21:58.351751    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc094f8f4_a12e_489e_92b3_908016822212.slice/crio-9fad240977e6d01331fe1a84b2467555b9dd81c9fb883a953d941f3d9cd874ec WatchSource:0}: Error finding container 9fad240977e6d01331fe1a84b2467555b9dd81c9fb883a953d941f3d9cd874ec: Status 404 returned error can't find the container with id 9fad240977e6d01331fe1a84b2467555b9dd81c9fb883a953d941f3d9cd874ec
Oct 30 01:21:58.352864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:58.352837    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"]
Oct 30 01:21:58.526782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:58.526743    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w" event={"ID":"c094f8f4-a12e-489e-92b3-908016822212","Type":"ContainerStarted","Data":"9fad240977e6d01331fe1a84b2467555b9dd81c9fb883a953d941f3d9cd874ec"}
Oct 30 01:21:59.536066 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:59.536024    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w" event={"ID":"c094f8f4-a12e-489e-92b3-908016822212","Type":"ContainerStarted","Data":"332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61"}
Oct 30 01:21:59.536520 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:59.536095    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:21:59.580043 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:21:59.579988    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w" podStartSLOduration=2.284301864 podStartE2EDuration="2.579974076s" totalImagesPullingTime="295.672212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:21:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:21:59.57616304 +0000 UTC m=+2561.678917010" watchObservedRunningTime="2026-10-30 01:21:59.579974076 +0000 UTC m=+2561.682728046"
Oct 30 01:22:05.550884 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:05.550851    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:22:14.924832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.924800    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"]
Oct 30 01:22:14.929692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.929667    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:14.945841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.945812    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"]
Oct 30 01:22:14.953343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.953290    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-96bd13-serving-cert\""
Oct 30 01:22:14.997521 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.997480    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-maas-api-tls\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:14.997737 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.997677    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-metrics-certs\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:14.997825 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.997791    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac84a502-752f-4b57-a13e-020a191f0d09-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:14.998083 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:14.998060    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dlt6\" (UniqueName: \"kubernetes.io/projected/ac84a502-752f-4b57-a13e-020a191f0d09-kube-api-access-7dlt6\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.099548 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.099504    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dlt6\" (UniqueName: \"kubernetes.io/projected/ac84a502-752f-4b57-a13e-020a191f0d09-kube-api-access-7dlt6\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.099548 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.099559    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-maas-api-tls\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.099852 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.099679    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-metrics-certs\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.099852 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.099771    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac84a502-752f-4b57-a13e-020a191f0d09-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.100556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.100528    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac84a502-752f-4b57-a13e-020a191f0d09-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.102741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.102717    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-maas-api-tls\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.102826 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.102735    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-metrics-certs\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.120026 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.119994    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dlt6\" (UniqueName: \"kubernetes.io/projected/ac84a502-752f-4b57-a13e-020a191f0d09-kube-api-access-7dlt6\") pod \"maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.253156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.253060    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:15.570230 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:22:15.570196    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac84a502_752f_4b57_a13e_020a191f0d09.slice/crio-bd4f24ffcafc206a744db8809bf338bcee7b884666b7524147a2b31078052980 WatchSource:0}: Error finding container bd4f24ffcafc206a744db8809bf338bcee7b884666b7524147a2b31078052980: Status 404 returned error can't find the container with id bd4f24ffcafc206a744db8809bf338bcee7b884666b7524147a2b31078052980
Oct 30 01:22:15.570940 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.570863    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"]
Oct 30 01:22:15.712007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:15.711938    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5" event={"ID":"ac84a502-752f-4b57-a13e-020a191f0d09","Type":"ContainerStarted","Data":"bd4f24ffcafc206a744db8809bf338bcee7b884666b7524147a2b31078052980"}
Oct 30 01:22:16.363838 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.363802    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"]
Oct 30 01:22:16.364365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.364073    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j" podUID="405af7db-c0a0-410a-b23f-df70bd1fe1d3" containerName="maas-api" containerID="cri-o://67d8bcbac621cf9e778fabb20a21666212e262eb70da73d99d358b4b4897ffea" gracePeriod=30
Oct 30 01:22:16.865504 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.861730    6011 generic.go:356] "Generic (PLEG): container finished" podID="405af7db-c0a0-410a-b23f-df70bd1fe1d3" containerID="67d8bcbac621cf9e778fabb20a21666212e262eb70da73d99d358b4b4897ffea" exitCode=0
Oct 30 01:22:16.865504 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.861877    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j" event={"ID":"405af7db-c0a0-410a-b23f-df70bd1fe1d3","Type":"ContainerDied","Data":"67d8bcbac621cf9e778fabb20a21666212e262eb70da73d99d358b4b4897ffea"}
Oct 30 01:22:16.884284 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.880217    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5" event={"ID":"ac84a502-752f-4b57-a13e-020a191f0d09","Type":"ContainerStarted","Data":"8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7"}
Oct 30 01:22:16.884284 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.881599    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:16.887710 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.887679    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:22:16.925003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.924939    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5" podStartSLOduration=2.487089563 podStartE2EDuration="2.92492507s" totalImagesPullingTime="437.835507ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:22:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:22:16.922170013 +0000 UTC m=+2579.024939605" watchObservedRunningTime="2026-10-30 01:22:16.92492507 +0000 UTC m=+2579.027679040"
Oct 30 01:22:16.966706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.966188    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/405af7db-c0a0-410a-b23f-df70bd1fe1d3-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/405af7db-c0a0-410a-b23f-df70bd1fe1d3-openshift-service-ca\") pod \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") "
Oct 30 01:22:16.966706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.966331    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-maas-api-tls\") pod \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") "
Oct 30 01:22:16.966706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.966381    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-metrics-certs\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-metrics-certs\") pod \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") "
Oct 30 01:22:16.966706 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.966565    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/405af7db-c0a0-410a-b23f-df70bd1fe1d3-kube-api-access-t85hn\" (UniqueName: \"kubernetes.io/projected/405af7db-c0a0-410a-b23f-df70bd1fe1d3-kube-api-access-t85hn\") pod \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\" (UID: \"405af7db-c0a0-410a-b23f-df70bd1fe1d3\") "
Oct 30 01:22:16.976857 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.976818    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/405af7db-c0a0-410a-b23f-df70bd1fe1d3-openshift-service-ca" pod "405af7db-c0a0-410a-b23f-df70bd1fe1d3" (UID: "405af7db-c0a0-410a-b23f-df70bd1fe1d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:22:16.979324 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.977597    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-maas-api-tls" pod "405af7db-c0a0-410a-b23f-df70bd1fe1d3" (UID: "405af7db-c0a0-410a-b23f-df70bd1fe1d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:22:16.979324 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.978627    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-metrics-certs" pod "405af7db-c0a0-410a-b23f-df70bd1fe1d3" (UID: "405af7db-c0a0-410a-b23f-df70bd1fe1d3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:22:16.990637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:16.990218    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/405af7db-c0a0-410a-b23f-df70bd1fe1d3-kube-api-access-t85hn" pod "405af7db-c0a0-410a-b23f-df70bd1fe1d3" (UID: "405af7db-c0a0-410a-b23f-df70bd1fe1d3"). InnerVolumeSpecName "kube-api-access-t85hn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:22:17.070488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.070438    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/405af7db-c0a0-410a-b23f-df70bd1fe1d3-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:17.070488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.070477    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:17.070488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.070491    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/405af7db-c0a0-410a-b23f-df70bd1fe1d3-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:17.070834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.070504    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t85hn\" (UniqueName: \"kubernetes.io/projected/405af7db-c0a0-410a-b23f-df70bd1fe1d3-kube-api-access-t85hn\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:17.890626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.890591    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"
Oct 30 01:22:17.890626 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.890601    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j" event={"ID":"405af7db-c0a0-410a-b23f-df70bd1fe1d3","Type":"ContainerDied","Data":"79fe4dab17824196fe878f115dc52c70a6bdfeff27f7083605df8c7bdb3c00d3"}
Oct 30 01:22:17.891285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.890673    6011 scope.go:122] "RemoveContainer" containerID="67d8bcbac621cf9e778fabb20a21666212e262eb70da73d99d358b4b4897ffea"
Oct 30 01:22:17.948453 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.948418    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"]
Oct 30 01:22:17.961340 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:17.961305    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w0-43d152-5cd7577569-vgm2j"]
Oct 30 01:22:18.024914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:18.024878    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="405af7db-c0a0-410a-b23f-df70bd1fe1d3" path="/var/lib/kubelet/pods/405af7db-c0a0-410a-b23f-df70bd1fe1d3/volumes"
Oct 30 01:22:23.908949 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:23.908920    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:22:27.916544 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:22:27.916504    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:22:29.479012 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:29.478980    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"]
Oct 30 01:22:35.101351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:35.101315    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"]
Oct 30 01:22:35.135236 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:35.135166    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"]
Oct 30 01:22:40.101722 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:40.101662    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" podUID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" containerName="payload-processing" containerID="cri-o://ceff8b422050a05b4722f832cdc7bac3a58e1c97634360d28dc4300cde569557" gracePeriod=25
Oct 30 01:22:40.135654 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:40.135605    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" podUID="2ff68182-b017-41ea-890f-40309bb8c4c4" containerName="payload-pre-processing" containerID="cri-o://a51764f7d3477c462489d0727108ca1d87ed2a7f1f03e7cbbf047b5898768bf2" gracePeriod=25
Oct 30 01:22:40.717420 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:40.717374    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" podUID="2ff68182-b017-41ea-890f-40309bb8c4c4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:22:40.717929 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:40.717598    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" podUID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:22:44.503411 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.503347    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main" containerID="cri-o://26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12" gracePeriod=45
Oct 30 01:22:44.785705 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.785675    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 01:22:44.830999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.830959    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tls-certs\" (UniqueName: \"kubernetes.io/secret/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tls-certs\") pod \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") "
Oct 30 01:22:44.831224 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.831025    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-home\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-home\") pod \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") "
Oct 30 01:22:44.831508 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.831449    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-home" pod "1622311a-a544-4ca5-bb79-ed4c0945b8bf" (UID: "1622311a-a544-4ca5-bb79-ed4c0945b8bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:22:44.833211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.831244    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1622311a-a544-4ca5-bb79-ed4c0945b8bf-kube-api-access-qrpmc\" (UniqueName: \"kubernetes.io/projected/1622311a-a544-4ca5-bb79-ed4c0945b8bf-kube-api-access-qrpmc\") pod \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") "
Oct 30 01:22:44.833689 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.833516    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tmp-dir\") pod \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") "
Oct 30 01:22:44.833689 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.833560    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-model-cache\") pod \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") "
Oct 30 01:22:44.833689 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.833615    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-dshm\") pod \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\" (UID: \"1622311a-a544-4ca5-bb79-ed4c0945b8bf\") "
Oct 30 01:22:44.833939 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.833759    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tmp-dir" pod "1622311a-a544-4ca5-bb79-ed4c0945b8bf" (UID: "1622311a-a544-4ca5-bb79-ed4c0945b8bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:22:44.833939 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.833904    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-model-cache" pod "1622311a-a544-4ca5-bb79-ed4c0945b8bf" (UID: "1622311a-a544-4ca5-bb79-ed4c0945b8bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:22:44.834057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.834041    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tls-certs" pod "1622311a-a544-4ca5-bb79-ed4c0945b8bf" (UID: "1622311a-a544-4ca5-bb79-ed4c0945b8bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:22:44.834226 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.834168    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1622311a-a544-4ca5-bb79-ed4c0945b8bf-kube-api-access-qrpmc" pod "1622311a-a544-4ca5-bb79-ed4c0945b8bf" (UID: "1622311a-a544-4ca5-bb79-ed4c0945b8bf"). InnerVolumeSpecName "kube-api-access-qrpmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:22:44.834896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.834872    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:44.834962 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.834903    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:44.834962 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.834916    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1622311a-a544-4ca5-bb79-ed4c0945b8bf-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:44.834962 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.834929    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:44.834962 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.834942    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qrpmc\" (UniqueName: \"kubernetes.io/projected/1622311a-a544-4ca5-bb79-ed4c0945b8bf-kube-api-access-qrpmc\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:44.836164 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.836136    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-dshm" pod "1622311a-a544-4ca5-bb79-ed4c0945b8bf" (UID: "1622311a-a544-4ca5-bb79-ed4c0945b8bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:22:44.936983 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:44.936937    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1622311a-a544-4ca5-bb79-ed4c0945b8bf-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:22:45.137788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.137750    6011 generic.go:356] "Generic (PLEG): container finished" podID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerID="26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12" exitCode=0
Oct 30 01:22:45.138052 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.137810    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"
Oct 30 01:22:45.138052 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.137822    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" event={"ID":"1622311a-a544-4ca5-bb79-ed4c0945b8bf","Type":"ContainerDied","Data":"26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12"}
Oct 30 01:22:45.138052 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.137855    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr" event={"ID":"1622311a-a544-4ca5-bb79-ed4c0945b8bf","Type":"ContainerDied","Data":"118e52aafbcc3263459cd93c2e2f93eee6d176ba33f28ccafdfc1a8e9e003d4d"}
Oct 30 01:22:45.138052 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.137878    6011 scope.go:122] "RemoveContainer" containerID="26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12"
Oct 30 01:22:45.273915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.273493    6011 scope.go:122] "RemoveContainer" containerID="26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12"
Oct 30 01:22:45.274582 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:22:45.274475    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12\": container with ID starting with 26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12 not found: ID does not exist" containerID="26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12"
Oct 30 01:22:45.274582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.274517    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12"} err="failed to get container status \"26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12\": rpc error: code = NotFound desc = could not find container \"26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12\": container with ID starting with 26a8613419a4c6fc20c25ea7cb9b64fdd4008677d0b48339f492d77336c1cc12 not found: ID does not exist"
Oct 30 01:22:45.302188 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.302131    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"]
Oct 30 01:22:45.320798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.320748    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-3c2596/distinct-w0-3c2596-kserve-6df759769c-2zfhr"]
Oct 30 01:22:45.379382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.379341    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"]
Oct 30 01:22:45.380783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.380758    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main"
Oct 30 01:22:45.380783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.380783    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main"
Oct 30 01:22:45.380957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.380816    6011 cpu_manager.go:414] "containerMap: removing container" podUID="405af7db-c0a0-410a-b23f-df70bd1fe1d3" containerName="maas-api"
Oct 30 01:22:45.380957 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.380824    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="405af7db-c0a0-410a-b23f-df70bd1fe1d3" containerName="maas-api"
Oct 30 01:22:45.381064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.381020    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" containerName="main"
Oct 30 01:22:45.381064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.381038    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="405af7db-c0a0-410a-b23f-df70bd1fe1d3" containerName="maas-api"
Oct 30 01:22:45.387059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.387034    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.415663 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.415631    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-24d5ce-openshift-default-dockercfg-bwdz5\""
Oct 30 01:22:45.418698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.418668    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"]
Oct 30 01:22:45.467205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.466756    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs2nw\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-kube-api-access-zs2nw\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.466866    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-socket\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.467025    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-certs\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.467135    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-credential-socket\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467621 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.467227    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-data\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467621 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.467268    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0961b59d-9703-4409-b5ae-c099ca05f66e-istiod-ca-cert\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467621 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.467328    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-podinfo\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467621 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.467369    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-token\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.467621 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.467426    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-envoy\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.568882    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-credential-socket\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.568938    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-data\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.568977    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0961b59d-9703-4409-b5ae-c099ca05f66e-istiod-ca-cert\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.569014    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-podinfo\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.569040    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-token\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.569072    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-envoy\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.569116    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs2nw\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-kube-api-access-zs2nw\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.569201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.569164    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-socket\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.570131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.569447    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-certs\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.570131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.570086    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-certs\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.571001 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.570558    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0961b59d-9703-4409-b5ae-c099ca05f66e-istiod-ca-cert\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.571001 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.570664    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-socket\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.571001 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.570949    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-credential-socket\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.571284 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.571106    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-data\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.573462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.573410    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-envoy\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.575989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.575942    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-podinfo\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.594896 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.594856    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-token\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.595639 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.595591    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs2nw\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-kube-api-access-zs2nw\") pod \"e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") " pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:45.717831 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:45.717494    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:46.029600 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:46.029509    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1622311a-a544-4ca5-bb79-ed4c0945b8bf" path="/var/lib/kubelet/pods/1622311a-a544-4ca5-bb79-ed4c0945b8bf/volumes"
Oct 30 01:22:46.119556 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:22:46.119522    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0961b59d_9703_4409_b5ae_c099ca05f66e.slice/crio-91e78b0ba004229446190f3a9fa5faf600ed0719435d3a8c166aee0ef3229e92 WatchSource:0}: Error finding container 91e78b0ba004229446190f3a9fa5faf600ed0719435d3a8c166aee0ef3229e92: Status 404 returned error can't find the container with id 91e78b0ba004229446190f3a9fa5faf600ed0719435d3a8c166aee0ef3229e92
Oct 30 01:22:46.119967 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:46.119942    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"]
Oct 30 01:22:46.135721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:46.135073    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:22:46.135721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:46.135216    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:22:46.135721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:46.135263    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:22:46.160026 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:46.159982    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" event={"ID":"0961b59d-9703-4409-b5ae-c099ca05f66e","Type":"ContainerStarted","Data":"91e78b0ba004229446190f3a9fa5faf600ed0719435d3a8c166aee0ef3229e92"}
Oct 30 01:22:47.176937 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:47.176894    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" event={"ID":"0961b59d-9703-4409-b5ae-c099ca05f66e","Type":"ContainerStarted","Data":"8e6447c6c21853009a5245a6c9d0097a02d6ca3d2c79998dfde7040bf0f30819"}
Oct 30 01:22:47.257202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:47.239626    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podStartSLOduration=2.23960728 podStartE2EDuration="2.23960728s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:22:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:22:47.23548248 +0000 UTC m=+2609.338236451" watchObservedRunningTime="2026-10-30 01:22:47.23960728 +0000 UTC m=+2609.342361254"
Oct 30 01:22:47.719225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:47.719156    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:47.720609 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:47.720586    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 01:22:47.720709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:47.720637    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 01:22:48.719350 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:48.719308    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 01:22:48.719796 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:48.719378    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 01:22:49.719825 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:49.719785    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 01:22:49.720295 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:49.719844    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 01:22:50.711512 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:50.711464    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" podUID="2ff68182-b017-41ea-890f-40309bb8c4c4" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:22:50.711912 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:50.711636    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" podUID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:22:50.719204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:50.719155    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 01:22:50.719350 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:50.719235    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 01:22:51.719049 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:51.719008    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 01:22:51.719594 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:51.719068    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 01:22:52.816712 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:52.814800    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:52.816712 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:52.816594    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:52.820744 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:22:52.820653    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:22:57.918855 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:22:57.918805    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:23:00.320977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.320781    6011 generic.go:356] "Generic (PLEG): container finished" podID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" containerID="ceff8b422050a05b4722f832cdc7bac3a58e1c97634360d28dc4300cde569557" exitCode=0
Oct 30 01:23:00.320977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.320894    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" event={"ID":"76c52766-11e5-45c9-8eaa-7de21df9f9ab","Type":"ContainerDied","Data":"ceff8b422050a05b4722f832cdc7bac3a58e1c97634360d28dc4300cde569557"}
Oct 30 01:23:00.323380 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.323351    6011 generic.go:356] "Generic (PLEG): container finished" podID="2ff68182-b017-41ea-890f-40309bb8c4c4" containerID="a51764f7d3477c462489d0727108ca1d87ed2a7f1f03e7cbbf047b5898768bf2" exitCode=0
Oct 30 01:23:00.323576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.323563    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" event={"ID":"2ff68182-b017-41ea-890f-40309bb8c4c4","Type":"ContainerDied","Data":"a51764f7d3477c462489d0727108ca1d87ed2a7f1f03e7cbbf047b5898768bf2"}
Oct 30 01:23:00.468525 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.468497    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 01:23:00.472710 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.472679    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 01:23:00.578585 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.578526    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2ff68182-b017-41ea-890f-40309bb8c4c4-kube-api-access-smx6j\" (UniqueName: \"kubernetes.io/projected/2ff68182-b017-41ea-890f-40309bb8c4c4-kube-api-access-smx6j\") pod \"2ff68182-b017-41ea-890f-40309bb8c4c4\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") "
Oct 30 01:23:00.578882 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.578609    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/76c52766-11e5-45c9-8eaa-7de21df9f9ab-kube-api-access-2dbw6\" (UniqueName: \"kubernetes.io/projected/76c52766-11e5-45c9-8eaa-7de21df9f9ab-kube-api-access-2dbw6\") pod \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") "
Oct 30 01:23:00.578882 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.578758    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2ff68182-b017-41ea-890f-40309bb8c4c4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ff68182-b017-41ea-890f-40309bb8c4c4-plugins-config-volume\") pod \"2ff68182-b017-41ea-890f-40309bb8c4c4\" (UID: \"2ff68182-b017-41ea-890f-40309bb8c4c4\") "
Oct 30 01:23:00.579231 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.579202    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ff68182-b017-41ea-890f-40309bb8c4c4-plugins-config-volume" pod "2ff68182-b017-41ea-890f-40309bb8c4c4" (UID: "2ff68182-b017-41ea-890f-40309bb8c4c4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:00.579475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.579444    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76c52766-11e5-45c9-8eaa-7de21df9f9ab-plugins-config-volume" pod "76c52766-11e5-45c9-8eaa-7de21df9f9ab" (UID: "76c52766-11e5-45c9-8eaa-7de21df9f9ab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:00.579557 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.579489    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/76c52766-11e5-45c9-8eaa-7de21df9f9ab-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c52766-11e5-45c9-8eaa-7de21df9f9ab-plugins-config-volume\") pod \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\" (UID: \"76c52766-11e5-45c9-8eaa-7de21df9f9ab\") "
Oct 30 01:23:00.580348 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.580323    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2ff68182-b017-41ea-890f-40309bb8c4c4-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:00.580418 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.580356    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76c52766-11e5-45c9-8eaa-7de21df9f9ab-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:00.581672 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.581650    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ff68182-b017-41ea-890f-40309bb8c4c4-kube-api-access-smx6j" pod "2ff68182-b017-41ea-890f-40309bb8c4c4" (UID: "2ff68182-b017-41ea-890f-40309bb8c4c4"). InnerVolumeSpecName "kube-api-access-smx6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:00.581763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.581689    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76c52766-11e5-45c9-8eaa-7de21df9f9ab-kube-api-access-2dbw6" pod "76c52766-11e5-45c9-8eaa-7de21df9f9ab" (UID: "76c52766-11e5-45c9-8eaa-7de21df9f9ab"). InnerVolumeSpecName "kube-api-access-2dbw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:00.682084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.681990    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-smx6j\" (UniqueName: \"kubernetes.io/projected/2ff68182-b017-41ea-890f-40309bb8c4c4-kube-api-access-smx6j\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:00.682084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:00.682025    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dbw6\" (UniqueName: \"kubernetes.io/projected/76c52766-11e5-45c9-8eaa-7de21df9f9ab-kube-api-access-2dbw6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:01.337754 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.337716    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"
Oct 30 01:23:01.338282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.337743    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b" event={"ID":"76c52766-11e5-45c9-8eaa-7de21df9f9ab","Type":"ContainerDied","Data":"a61d85997b27ccc4e7052f8ae9dd6cabf37cbd699a4e3e4a9e6ea107dc47af71"}
Oct 30 01:23:01.338282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.337841    6011 scope.go:122] "RemoveContainer" containerID="ceff8b422050a05b4722f832cdc7bac3a58e1c97634360d28dc4300cde569557"
Oct 30 01:23:01.340104 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.339819    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j" event={"ID":"2ff68182-b017-41ea-890f-40309bb8c4c4","Type":"ContainerDied","Data":"843c65ccab00e6bdd3aaa5be34f3ee440939a9008c9654a2aece176720463932"}
Oct 30 01:23:01.340104 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.339936    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"
Oct 30 01:23:01.351965 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.351944    6011 scope.go:122] "RemoveContainer" containerID="a51764f7d3477c462489d0727108ca1d87ed2a7f1f03e7cbbf047b5898768bf2"
Oct 30 01:23:01.396193 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.396088    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"]
Oct 30 01:23:01.409406 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.409373    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-3c2596-849c477c95-h5p6j"]
Oct 30 01:23:01.446930 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.446888    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"]
Oct 30 01:23:01.450880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:01.450851    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-3c2596-67b774b75d-2ng8b"]
Oct 30 01:23:02.023070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:02.023034    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2ff68182-b017-41ea-890f-40309bb8c4c4" path="/var/lib/kubelet/pods/2ff68182-b017-41ea-890f-40309bb8c4c4/volumes"
Oct 30 01:23:02.023560 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:02.023543    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" path="/var/lib/kubelet/pods/76c52766-11e5-45c9-8eaa-7de21df9f9ab/volumes"
Oct 30 01:23:13.483486 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.483448    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"]
Oct 30 01:23:13.484235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.484197    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j" podUID="c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" containerName="maas-api" containerID="cri-o://9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78" gracePeriod=30
Oct 30 01:23:13.643473 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.643438    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"]
Oct 30 01:23:13.643832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.643704    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4" podUID="7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" containerName="payload-processing" containerID="cri-o://57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961" gracePeriod=30
Oct 30 01:23:13.659014 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.658983    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"]
Oct 30 01:23:13.660268 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.660232    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf" podUID="74046b45-fc09-463e-8034-7e0797071d15" containerName="payload-pre-processing" containerID="cri-o://389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3" gracePeriod=30
Oct 30 01:23:13.847760 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.847436    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:23:13.949862 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.949836    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:23:13.968563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.968528    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-maas-api-tls\") pod \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") "
Oct 30 01:23:13.968728 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.968680    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-kube-api-access-tg8wj\" (UniqueName: \"kubernetes.io/projected/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-kube-api-access-tg8wj\") pod \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") "
Oct 30 01:23:13.968801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.968723    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-metrics-certs\") pod \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") "
Oct 30 01:23:13.969270 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.969242    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-openshift-service-ca\") pod \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\" (UID: \"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d\") "
Oct 30 01:23:13.970230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.970208    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-openshift-service-ca" pod "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" (UID: "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:13.971717 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.971694    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-maas-api-tls" pod "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" (UID: "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:13.971826 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.971731    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-kube-api-access-tg8wj" pod "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" (UID: "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d"). InnerVolumeSpecName "kube-api-access-tg8wj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:13.972334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.972317    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-metrics-certs" pod "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" (UID: "c5f61aac-9ddd-47bc-86a8-7fd100a2e19d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:13.996570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:13.996536    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:23:14.195831 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.195795    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-kube-api-access-bxgsm\" (UniqueName: \"kubernetes.io/projected/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-kube-api-access-bxgsm\") pod \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") "
Oct 30 01:23:14.196438 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.196387    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-plugins-config-volume\") pod \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\" (UID: \"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d\") "
Oct 30 01:23:14.197386 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.197333    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg8wj\" (UniqueName: \"kubernetes.io/projected/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-kube-api-access-tg8wj\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.197530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.197512    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.197645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.197631    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.197748 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.197733    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.198250 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.198228    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-plugins-config-volume" pod "7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" (UID: "7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:14.204543 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.203405    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-kube-api-access-bxgsm" pod "7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" (UID: "7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d"). InnerVolumeSpecName "kube-api-access-bxgsm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:14.300771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.300723    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/74046b45-fc09-463e-8034-7e0797071d15-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74046b45-fc09-463e-8034-7e0797071d15-plugins-config-volume\") pod \"74046b45-fc09-463e-8034-7e0797071d15\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") "
Oct 30 01:23:14.301037 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.301015    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/74046b45-fc09-463e-8034-7e0797071d15-kube-api-access-h6d4h\" (UniqueName: \"kubernetes.io/projected/74046b45-fc09-463e-8034-7e0797071d15-kube-api-access-h6d4h\") pod \"74046b45-fc09-463e-8034-7e0797071d15\" (UID: \"74046b45-fc09-463e-8034-7e0797071d15\") "
Oct 30 01:23:14.302240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.302213    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.302346 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.302280    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxgsm\" (UniqueName: \"kubernetes.io/projected/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d-kube-api-access-bxgsm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.302675 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.302650    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74046b45-fc09-463e-8034-7e0797071d15-plugins-config-volume" pod "74046b45-fc09-463e-8034-7e0797071d15" (UID: "74046b45-fc09-463e-8034-7e0797071d15"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:14.305469 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.305434    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74046b45-fc09-463e-8034-7e0797071d15-kube-api-access-h6d4h" pod "74046b45-fc09-463e-8034-7e0797071d15" (UID: "74046b45-fc09-463e-8034-7e0797071d15"). InnerVolumeSpecName "kube-api-access-h6d4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:14.405389 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.405347    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6d4h\" (UniqueName: \"kubernetes.io/projected/74046b45-fc09-463e-8034-7e0797071d15-kube-api-access-h6d4h\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.405389 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.405388    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/74046b45-fc09-463e-8034-7e0797071d15-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:14.501313 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.501218    6011 generic.go:356] "Generic (PLEG): container finished" podID="c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" containerID="9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78" exitCode=0
Oct 30 01:23:14.501313 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.501300    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"
Oct 30 01:23:14.501805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.501301    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j" event={"ID":"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d","Type":"ContainerDied","Data":"9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78"}
Oct 30 01:23:14.501805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.501392    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j" event={"ID":"c5f61aac-9ddd-47bc-86a8-7fd100a2e19d","Type":"ContainerDied","Data":"c6af07b7a1a7742a912fb8376e329ee6c23710bb355aafe9a349b16aed2bbfb6"}
Oct 30 01:23:14.501805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.501428    6011 scope.go:122] "RemoveContainer" containerID="9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78"
Oct 30 01:23:14.502948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.502921    6011 generic.go:356] "Generic (PLEG): container finished" podID="74046b45-fc09-463e-8034-7e0797071d15" containerID="389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3" exitCode=0
Oct 30 01:23:14.503069 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.502945    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf" event={"ID":"74046b45-fc09-463e-8034-7e0797071d15","Type":"ContainerDied","Data":"389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3"}
Oct 30 01:23:14.503069 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.502988    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf" event={"ID":"74046b45-fc09-463e-8034-7e0797071d15","Type":"ContainerDied","Data":"4d4d228fa5c10b23f38d96bd5e2bf13a82ab012860a0f48ad37454bca6300f09"}
Oct 30 01:23:14.503069 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.502995    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"
Oct 30 01:23:14.505030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.505003    6011 generic.go:356] "Generic (PLEG): container finished" podID="7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" containerID="57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961" exitCode=0
Oct 30 01:23:14.505152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.505055    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4" event={"ID":"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d","Type":"ContainerDied","Data":"57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961"}
Oct 30 01:23:14.505152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.505097    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4" event={"ID":"7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d","Type":"ContainerDied","Data":"0ac6ad188be479189b22485efec3cbcb1548ad22f0cd76388155e23d93976338"}
Oct 30 01:23:14.505152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.505116    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"
Oct 30 01:23:14.514886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.514867    6011 scope.go:122] "RemoveContainer" containerID="9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78"
Oct 30 01:23:14.515230 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:14.515204    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78\": container with ID starting with 9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78 not found: ID does not exist" containerID="9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78"
Oct 30 01:23:14.515343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.515238    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78"} err="failed to get container status \"9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78\": rpc error: code = NotFound desc = could not find container \"9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78\": container with ID starting with 9ef38091fa42d09b058bd0778a84b1ac41b73ce0d1e5a613e74ba52007d17b78 not found: ID does not exist"
Oct 30 01:23:14.515343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.515256    6011 scope.go:122] "RemoveContainer" containerID="389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3"
Oct 30 01:23:14.546462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.546438    6011 scope.go:122] "RemoveContainer" containerID="389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3"
Oct 30 01:23:14.546804 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:14.546773    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3\": container with ID starting with 389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3 not found: ID does not exist" containerID="389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3"
Oct 30 01:23:14.546894 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.546816    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3"} err="failed to get container status \"389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3\": rpc error: code = NotFound desc = could not find container \"389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3\": container with ID starting with 389aecae06cd933acd668c839997719607a5550a341262e1f34e588afa10e8c3 not found: ID does not exist"
Oct 30 01:23:14.546894 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.546839    6011 scope.go:122] "RemoveContainer" containerID="57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961"
Oct 30 01:23:14.558096 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.558076    6011 scope.go:122] "RemoveContainer" containerID="57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961"
Oct 30 01:23:14.558394 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:14.558374    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961\": container with ID starting with 57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961 not found: ID does not exist" containerID="57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961"
Oct 30 01:23:14.558481 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.558400    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961"} err="failed to get container status \"57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961\": rpc error: code = NotFound desc = could not find container \"57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961\": container with ID starting with 57737ecdff0880f15cb5305c5b63ade644db405c5559d82484cbbcf79b410961 not found: ID does not exist"
Oct 30 01:23:14.559724 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.559708    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"]
Oct 30 01:23:14.564110 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.564075    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f461c3-69c89c7544-h9b7j"]
Oct 30 01:23:14.625036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.621993    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"]
Oct 30 01:23:14.638971 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.638144    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f461c3-59d94d9d8-cjtl4"]
Oct 30 01:23:14.659093 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.659057    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"]
Oct 30 01:23:14.666589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:14.666560    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f461c3-74f94d7c95-sgdjf"]
Oct 30 01:23:16.023552 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:16.023518    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="74046b45-fc09-463e-8034-7e0797071d15" path="/var/lib/kubelet/pods/74046b45-fc09-463e-8034-7e0797071d15/volumes"
Oct 30 01:23:16.024023 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:16.024008    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" path="/var/lib/kubelet/pods/7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d/volumes"
Oct 30 01:23:16.024511 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:16.024495    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" path="/var/lib/kubelet/pods/c5f61aac-9ddd-47bc-86a8-7fd100a2e19d/volumes"
Oct 30 01:23:19.954379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:19.954341    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"]
Oct 30 01:23:19.954833 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:19.954614    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw" podUID="98692fcb-68d5-455c-8379-46033ebdf716" containerName="maas-api" containerID="cri-o://1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179" gracePeriod=30
Oct 30 01:23:20.150484 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.150449    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"]
Oct 30 01:23:20.150732 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.150697    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7" podUID="f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" containerName="payload-pre-processing" containerID="cri-o://e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95" gracePeriod=30
Oct 30 01:23:20.301349 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.301321    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:23:20.401354 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.401301    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-maas-api-tls\") pod \"98692fcb-68d5-455c-8379-46033ebdf716\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") "
Oct 30 01:23:20.401553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.401363    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/98692fcb-68d5-455c-8379-46033ebdf716-kube-api-access-65vd4\" (UniqueName: \"kubernetes.io/projected/98692fcb-68d5-455c-8379-46033ebdf716-kube-api-access-65vd4\") pod \"98692fcb-68d5-455c-8379-46033ebdf716\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") "
Oct 30 01:23:20.401553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.401414    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-metrics-certs\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-metrics-certs\") pod \"98692fcb-68d5-455c-8379-46033ebdf716\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") "
Oct 30 01:23:20.401553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.401459    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/98692fcb-68d5-455c-8379-46033ebdf716-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98692fcb-68d5-455c-8379-46033ebdf716-openshift-service-ca\") pod \"98692fcb-68d5-455c-8379-46033ebdf716\" (UID: \"98692fcb-68d5-455c-8379-46033ebdf716\") "
Oct 30 01:23:20.402906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.402873    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98692fcb-68d5-455c-8379-46033ebdf716-openshift-service-ca" pod "98692fcb-68d5-455c-8379-46033ebdf716" (UID: "98692fcb-68d5-455c-8379-46033ebdf716"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:20.404062 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.404038    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-maas-api-tls" pod "98692fcb-68d5-455c-8379-46033ebdf716" (UID: "98692fcb-68d5-455c-8379-46033ebdf716"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:20.404483 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.404461    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98692fcb-68d5-455c-8379-46033ebdf716-kube-api-access-65vd4" pod "98692fcb-68d5-455c-8379-46033ebdf716" (UID: "98692fcb-68d5-455c-8379-46033ebdf716"). InnerVolumeSpecName "kube-api-access-65vd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:20.406324 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.406298    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-metrics-certs" pod "98692fcb-68d5-455c-8379-46033ebdf716" (UID: "98692fcb-68d5-455c-8379-46033ebdf716"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:20.428106 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.428080    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:23:20.523276 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.510773    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-kube-api-access-596d4\" (UniqueName: \"kubernetes.io/projected/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-kube-api-access-596d4\") pod \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") "
Oct 30 01:23:20.523276 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.511031    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-plugins-config-volume\") pod \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\" (UID: \"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5\") "
Oct 30 01:23:20.531290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.526412    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:20.531290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.526452    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-65vd4\" (UniqueName: \"kubernetes.io/projected/98692fcb-68d5-455c-8379-46033ebdf716-kube-api-access-65vd4\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:20.531290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.526467    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98692fcb-68d5-455c-8379-46033ebdf716-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:20.531290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.526480    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98692fcb-68d5-455c-8379-46033ebdf716-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:20.531290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.526993    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-plugins-config-volume" pod "f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" (UID: "f27f286f-4c8a-4a2b-a99f-c85119b8ebd5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:20.531290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.531073    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-kube-api-access-596d4" pod "f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" (UID: "f27f286f-4c8a-4a2b-a99f-c85119b8ebd5"). InnerVolumeSpecName "kube-api-access-596d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:20.680451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.680373    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:20.680451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.680421    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-596d4\" (UniqueName: \"kubernetes.io/projected/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5-kube-api-access-596d4\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:20.714199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.700890    6011 generic.go:356] "Generic (PLEG): container finished" podID="98692fcb-68d5-455c-8379-46033ebdf716" containerID="1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179" exitCode=0
Oct 30 01:23:20.714199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.701015    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw" event={"ID":"98692fcb-68d5-455c-8379-46033ebdf716","Type":"ContainerDied","Data":"1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179"}
Oct 30 01:23:20.714199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.701071    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw" event={"ID":"98692fcb-68d5-455c-8379-46033ebdf716","Type":"ContainerDied","Data":"b38e6ae53915e64926b446e40ba577e6613c2699528af796a5fffdc14fb2965b"}
Oct 30 01:23:20.714199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.701110    6011 scope.go:122] "RemoveContainer" containerID="1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179"
Oct 30 01:23:20.714199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.701405    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"
Oct 30 01:23:20.722788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.721225    6011 generic.go:356] "Generic (PLEG): container finished" podID="f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" containerID="e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95" exitCode=0
Oct 30 01:23:20.722788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.721374    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7" event={"ID":"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5","Type":"ContainerDied","Data":"e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95"}
Oct 30 01:23:20.722788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.721448    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7" event={"ID":"f27f286f-4c8a-4a2b-a99f-c85119b8ebd5","Type":"ContainerDied","Data":"727eb58e256f89f3ac2452f54eb3b4f67c3f1de535bd4e719ccaeb719cacb163"}
Oct 30 01:23:20.722788 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.721575    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"
Oct 30 01:23:20.741235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.741211    6011 scope.go:122] "RemoveContainer" containerID="1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179"
Oct 30 01:23:20.741578 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:20.741553    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179\": container with ID starting with 1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179 not found: ID does not exist" containerID="1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179"
Oct 30 01:23:20.741665 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.741585    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179"} err="failed to get container status \"1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179\": rpc error: code = NotFound desc = could not find container \"1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179\": container with ID starting with 1d36e6c5d5e02b8dd57133437680a6200237fb329718f609daf8c721e853e179 not found: ID does not exist"
Oct 30 01:23:20.741665 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.741605    6011 scope.go:122] "RemoveContainer" containerID="e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95"
Oct 30 01:23:20.752646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.752620    6011 scope.go:122] "RemoveContainer" containerID="e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95"
Oct 30 01:23:20.752922 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:20.752901    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95\": container with ID starting with e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95 not found: ID does not exist" containerID="e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95"
Oct 30 01:23:20.753003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.752929    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95"} err="failed to get container status \"e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95\": rpc error: code = NotFound desc = could not find container \"e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95\": container with ID starting with e64ad492480087a182a479c14067d64573308aa765fadab8e4531e5ae5001b95 not found: ID does not exist"
Oct 30 01:23:20.783028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.782989    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"]
Oct 30 01:23:20.790695 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.790667    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-1075cb-665656cc8f-9clhw"]
Oct 30 01:23:20.839823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.839787    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"]
Oct 30 01:23:20.846016 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:20.845986    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-1075cb-85f95b57f9-zb2k7"]
Oct 30 01:23:22.023235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:22.023160    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="98692fcb-68d5-455c-8379-46033ebdf716" path="/var/lib/kubelet/pods/98692fcb-68d5-455c-8379-46033ebdf716/volumes"
Oct 30 01:23:22.023709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:22.023692    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" path="/var/lib/kubelet/pods/f27f286f-4c8a-4a2b-a99f-c85119b8ebd5/volumes"
Oct 30 01:23:27.965784 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:27.965727    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:23:33.884910 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:33.884875    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"]
Oct 30 01:23:33.885608 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:33.885127    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg" podUID="58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" containerName="maas-api" containerID="cri-o://f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643" gracePeriod=30
Oct 30 01:23:34.198556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.198526    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:23:34.298119 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.298073    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"]
Oct 30 01:23:34.298401 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.298334    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr" podUID="2d360ea0-4cc2-4236-b5df-e3dd6369712a" containerName="maas-api" containerID="cri-o://8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16" gracePeriod=30
Oct 30 01:23:34.308025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.307991    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-openshift-service-ca\") pod \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") "
Oct 30 01:23:34.308314 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.308265    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-metrics-certs\") pod \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") "
Oct 30 01:23:34.308400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.308332    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-maas-api-tls\") pod \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") "
Oct 30 01:23:34.308400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.308376    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-kube-api-access-2dpjh\" (UniqueName: \"kubernetes.io/projected/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-kube-api-access-2dpjh\") pod \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\" (UID: \"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99\") "
Oct 30 01:23:34.309734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.309694    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-openshift-service-ca" pod "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" (UID: "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:34.313143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.313115    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-maas-api-tls" pod "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" (UID: "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:34.313378 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.313354    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-kube-api-access-2dpjh" pod "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" (UID: "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99"). InnerVolumeSpecName "kube-api-access-2dpjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:34.314068 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.314048    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-metrics-certs" pod "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" (UID: "58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:34.410776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.410694    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.410776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.410732    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.410776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.410742    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.410776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.410751    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dpjh\" (UniqueName: \"kubernetes.io/projected/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99-kube-api-access-2dpjh\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.604314 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.604283    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:23:34.716518 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.716433    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2d360ea0-4cc2-4236-b5df-e3dd6369712a-kube-api-access-m52fd\" (UniqueName: \"kubernetes.io/projected/2d360ea0-4cc2-4236-b5df-e3dd6369712a-kube-api-access-m52fd\") pod \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") "
Oct 30 01:23:34.716518 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.716479    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-metrics-certs\") pod \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") "
Oct 30 01:23:34.716757 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.716591    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-maas-api-tls\") pod \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") "
Oct 30 01:23:34.716829 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.716809    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2d360ea0-4cc2-4236-b5df-e3dd6369712a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d360ea0-4cc2-4236-b5df-e3dd6369712a-openshift-service-ca\") pod \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\" (UID: \"2d360ea0-4cc2-4236-b5df-e3dd6369712a\") "
Oct 30 01:23:34.717383 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.717266    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d360ea0-4cc2-4236-b5df-e3dd6369712a-openshift-service-ca" pod "2d360ea0-4cc2-4236-b5df-e3dd6369712a" (UID: "2d360ea0-4cc2-4236-b5df-e3dd6369712a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:34.718012 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.717982    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d360ea0-4cc2-4236-b5df-e3dd6369712a-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.719048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.719024    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d360ea0-4cc2-4236-b5df-e3dd6369712a-kube-api-access-m52fd" pod "2d360ea0-4cc2-4236-b5df-e3dd6369712a" (UID: "2d360ea0-4cc2-4236-b5df-e3dd6369712a"). InnerVolumeSpecName "kube-api-access-m52fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:34.719048 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.719034    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-maas-api-tls" pod "2d360ea0-4cc2-4236-b5df-e3dd6369712a" (UID: "2d360ea0-4cc2-4236-b5df-e3dd6369712a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:34.719246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.719231    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-metrics-certs" pod "2d360ea0-4cc2-4236-b5df-e3dd6369712a" (UID: "2d360ea0-4cc2-4236-b5df-e3dd6369712a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:34.837239 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.824648    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.837239 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.824694    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m52fd\" (UniqueName: \"kubernetes.io/projected/2d360ea0-4cc2-4236-b5df-e3dd6369712a-kube-api-access-m52fd\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.837239 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.824710    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2d360ea0-4cc2-4236-b5df-e3dd6369712a-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.974439    6011 generic.go:356] "Generic (PLEG): container finished" podID="58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" containerID="f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643" exitCode=0
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.974501    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.974694    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg" event={"ID":"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99","Type":"ContainerDied","Data":"f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643"}
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.974731    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg" event={"ID":"58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99","Type":"ContainerDied","Data":"2f3544c165b109df2e484f1fe64dbc870a4e172a025e237ea6afaf240d467154"}
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.974759    6011 scope.go:122] "RemoveContainer" containerID="f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643"
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.977258    6011 generic.go:356] "Generic (PLEG): container finished" podID="2d360ea0-4cc2-4236-b5df-e3dd6369712a" containerID="8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16" exitCode=0
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.977347    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr" event={"ID":"2d360ea0-4cc2-4236-b5df-e3dd6369712a","Type":"ContainerDied","Data":"8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16"}
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.977394    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr" event={"ID":"2d360ea0-4cc2-4236-b5df-e3dd6369712a","Type":"ContainerDied","Data":"e02c0f893902a9cde1e67f138b92777f13504596fcdac0322b2410d8bca633d9"}
Oct 30 01:23:34.980265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.977455    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"
Oct 30 01:23:34.995906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.995874    6011 scope.go:122] "RemoveContainer" containerID="f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643"
Oct 30 01:23:34.996349 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:34.996324    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643\": container with ID starting with f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643 not found: ID does not exist" containerID="f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643"
Oct 30 01:23:34.996482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.996356    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643"} err="failed to get container status \"f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643\": rpc error: code = NotFound desc = could not find container \"f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643\": container with ID starting with f4a4821f9179b50c803e27530342c2641a3abaf1368be867deae9e9da8996643 not found: ID does not exist"
Oct 30 01:23:34.996482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:34.996373    6011 scope.go:122] "RemoveContainer" containerID="8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16"
Oct 30 01:23:35.013312 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:35.013288    6011 scope.go:122] "RemoveContainer" containerID="8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16"
Oct 30 01:23:35.013692 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:35.013667    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16\": container with ID starting with 8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16 not found: ID does not exist" containerID="8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16"
Oct 30 01:23:35.013806 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:35.013703    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16"} err="failed to get container status \"8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16\": rpc error: code = NotFound desc = could not find container \"8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16\": container with ID starting with 8d5c47747b71f674397bb953448e06655f8217c2c16214b1a6027215034c9d16 not found: ID does not exist"
Oct 30 01:23:35.033799 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:35.033766    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"]
Oct 30 01:23:35.047206 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:35.047147    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-5c515e-7cb6df5cf7-9nffr"]
Oct 30 01:23:35.070970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:35.070940    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"]
Oct 30 01:23:35.074431 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:35.074402    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-07a3a8-58bbb8c6f4-2hzbg"]
Oct 30 01:23:36.023865 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:36.023828    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2d360ea0-4cc2-4236-b5df-e3dd6369712a" path="/var/lib/kubelet/pods/2d360ea0-4cc2-4236-b5df-e3dd6369712a/volumes"
Oct 30 01:23:36.024614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:36.024501    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" path="/var/lib/kubelet/pods/58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99/volumes"
Oct 30 01:23:36.428086 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:36.428047    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"]
Oct 30 01:23:41.156670 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:41.156630    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"]
Oct 30 01:23:41.159393 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:41.156929    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy" containerID="cri-o://153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3" gracePeriod=30
Oct 30 01:23:41.168187 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:41.168145    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"]
Oct 30 01:23:41.168462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:41.168415    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy" containerID="cri-o://c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436" gracePeriod=30
Oct 30 01:23:41.428918 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:41.428816    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" podUID="cffdfd57-7d46-40f0-89c3-5d74d036b541" containerName="payload-processing" containerID="cri-o://85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b" gracePeriod=25
Oct 30 01:23:42.515764 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:42.515724    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"]
Oct 30 01:23:42.516536 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:42.515993    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy" containerID="cri-o://38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25" gracePeriod=30
Oct 30 01:23:42.530849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:42.530807    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"]
Oct 30 01:23:42.531441 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:42.531401    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy" containerID="cri-o://e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a" gracePeriod=30
Oct 30 01:23:42.536743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:42.536708    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"]
Oct 30 01:23:42.536951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:42.536928    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy" containerID="cri-o://ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03" gracePeriod=30
Oct 30 01:23:44.435298 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.435259    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"]
Oct 30 01:23:44.435973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.435958    6011 cpu_manager.go:414] "containerMap: removing container" podUID="2d360ea0-4cc2-4236-b5df-e3dd6369712a" containerName="maas-api"
Oct 30 01:23:44.435973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.435973    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2d360ea0-4cc2-4236-b5df-e3dd6369712a" containerName="maas-api"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.435989    6011 cpu_manager.go:414] "containerMap: removing container" podUID="58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" containerName="maas-api"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.435994    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" containerName="maas-api"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436007    6011 cpu_manager.go:414] "containerMap: removing container" podUID="2ff68182-b017-41ea-890f-40309bb8c4c4" containerName="payload-pre-processing"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436012    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2ff68182-b017-41ea-890f-40309bb8c4c4" containerName="payload-pre-processing"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436023    6011 cpu_manager.go:414] "containerMap: removing container" podUID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" containerName="payload-processing"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436027    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" containerName="payload-processing"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436039    6011 cpu_manager.go:414] "containerMap: removing container" podUID="7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" containerName="payload-processing"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436044    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" containerName="payload-processing"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436055    6011 cpu_manager.go:414] "containerMap: removing container" podUID="74046b45-fc09-463e-8034-7e0797071d15" containerName="payload-pre-processing"
Oct 30 01:23:44.436059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436060    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="74046b45-fc09-463e-8034-7e0797071d15" containerName="payload-pre-processing"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436075    6011 cpu_manager.go:414] "containerMap: removing container" podUID="f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" containerName="payload-pre-processing"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436081    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" containerName="payload-pre-processing"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436094    6011 cpu_manager.go:414] "containerMap: removing container" podUID="c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436099    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436113    6011 cpu_manager.go:414] "containerMap: removing container" podUID="98692fcb-68d5-455c-8379-46033ebdf716" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436118    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="98692fcb-68d5-455c-8379-46033ebdf716" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436242    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="f27f286f-4c8a-4a2b-a99f-c85119b8ebd5" containerName="payload-pre-processing"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436253    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5f61aac-9ddd-47bc-86a8-7fd100a2e19d" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436261    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="98692fcb-68d5-455c-8379-46033ebdf716" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436267    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="2d360ea0-4cc2-4236-b5df-e3dd6369712a" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436275    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="58c9ce8d-be5c-46b1-8b90-3f8a3b75fc99" containerName="maas-api"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436281    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="2ff68182-b017-41ea-890f-40309bb8c4c4" containerName="payload-pre-processing"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436287    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="76c52766-11e5-45c9-8eaa-7de21df9f9ab" containerName="payload-processing"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436293    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fafbcc8-ddc6-4fd7-bd0a-f1fb67657f2d" containerName="payload-processing"
Oct 30 01:23:44.436415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.436300    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="74046b45-fc09-463e-8034-7e0797071d15" containerName="payload-pre-processing"
Oct 30 01:23:44.440107 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.440087    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.457217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.457167    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"]
Oct 30 01:23:44.465377 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.465344    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-11e7a9-serving-cert\""
Oct 30 01:23:44.594699 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.594665    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs66w\" (UniqueName: \"kubernetes.io/projected/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-kube-api-access-hs66w\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.594876 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.594715    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-metrics-certs\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.594924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.594892    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.595111 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.595092    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.696521 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.696431    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.696521 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.696516    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs66w\" (UniqueName: \"kubernetes.io/projected/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-kube-api-access-hs66w\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.696765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.696546    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-metrics-certs\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.696765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.696670    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.699834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.699780    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.700094 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.700070    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.700629 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.700603    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-metrics-certs\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.720059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.720029    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs66w\" (UniqueName: \"kubernetes.io/projected/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-kube-api-access-hs66w\") pod \"maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:44.760953 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:44.756719    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:45.160110 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:23:45.160079    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb35586e_1bee_4d3a_8d7c_d4f66d9f595d.slice/crio-783833ca9156d8851b68d41bc4ecf62bc0b89d21cd8d4768caf40332fc7cdf41 WatchSource:0}: Error finding container 783833ca9156d8851b68d41bc4ecf62bc0b89d21cd8d4768caf40332fc7cdf41: Status 404 returned error can't find the container with id 783833ca9156d8851b68d41bc4ecf62bc0b89d21cd8d4768caf40332fc7cdf41
Oct 30 01:23:45.161538 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:45.161510    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"]
Oct 30 01:23:45.161981 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:45.161964    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:23:46.066331 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.066292    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc" event={"ID":"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d","Type":"ContainerStarted","Data":"2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c"}
Oct 30 01:23:46.066740 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.066359    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc" event={"ID":"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d","Type":"ContainerStarted","Data":"783833ca9156d8851b68d41bc4ecf62bc0b89d21cd8d4768caf40332fc7cdf41"}
Oct 30 01:23:46.066740 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.066390    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:46.110881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.110816    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc" podStartSLOduration=1.7545539799999998 podStartE2EDuration="2.110793719s" totalImagesPullingTime="356.239739ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:23:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:23:46.108700635 +0000 UTC m=+2668.211454606" watchObservedRunningTime="2026-10-30 01:23:46.110793719 +0000 UTC m=+2668.213547691"
Oct 30 01:23:46.511082 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.511055    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:23:46.517078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.517050    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:23:46.650825 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.650714    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-podinfo\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.650825 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.650783    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-podinfo\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.650869    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-credential-socket\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.650917    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-token\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-token\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.650945    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-token\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-token\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.650999    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-envoy\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651040    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-socket\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651125    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-socket\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651210    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-certs\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651309    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-kube-api-access-vws6l\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-kube-api-access-vws6l\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651314    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-credential-socket" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.651628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651397    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istiod-ca-cert\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651442    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-data\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651475    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-envoy\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651519    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-credential-socket\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651604    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-socket" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.651905 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651630    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-certs\") pod \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\" (UID: \"6edb9b4d-ba8c-4679-8ee4-a36537ac205e\") "
Oct 30 01:23:46.651905 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651680    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-data\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651905 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651714    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-kube-api-access-9dxbl\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-kube-api-access-9dxbl\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651905 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651754    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b92022e8-a46c-45cd-a161-cad0a4af03f0-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b92022e8-a46c-45cd-a161-cad0a4af03f0-istiod-ca-cert\") pod \"b92022e8-a46c-45cd-a161-cad0a4af03f0\" (UID: \"b92022e8-a46c-45cd-a161-cad0a4af03f0\") "
Oct 30 01:23:46.651905 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.651837    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-socket" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.652393 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.652347    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istiod-ca-cert" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.652649    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-certs" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.652703    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b92022e8-a46c-45cd-a161-cad0a4af03f0-istiod-ca-cert" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.652972    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-data" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.653052    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.653076    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.653094    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.653108    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.653325    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-certs" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.653503    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-credential-socket" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.653945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.653685    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-data" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.655745 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.655712    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-podinfo" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:23:46.656023 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.655994    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-envoy" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.656475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.656441    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-podinfo" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:23:46.656751 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.656717    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-envoy" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:46.657075 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.657046    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-token" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:46.657168 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.657122    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-kube-api-access-vws6l" pod "6edb9b4d-ba8c-4679-8ee4-a36537ac205e" (UID: "6edb9b4d-ba8c-4679-8ee4-a36537ac205e"). InnerVolumeSpecName "kube-api-access-vws6l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:46.657168 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.657139    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-kube-api-access-9dxbl" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "kube-api-access-9dxbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:46.657559 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.657539    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-token" pod "b92022e8-a46c-45cd-a161-cad0a4af03f0" (UID: "b92022e8-a46c-45cd-a161-cad0a4af03f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:46.754614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754574    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754617    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vws6l\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-kube-api-access-vws6l\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754636    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754651    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754664    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754678    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754691    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754703    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dxbl\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-kube-api-access-9dxbl\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754719    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b92022e8-a46c-45cd-a161-cad0a4af03f0-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754732    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754746    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754758    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b92022e8-a46c-45cd-a161-cad0a4af03f0-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.754776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754772    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:46.755603 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:46.754784    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6edb9b4d-ba8c-4679-8ee4-a36537ac205e-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.075542 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.075509    6011 generic.go:356] "Generic (PLEG): container finished" podID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerID="c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436" exitCode=0
Oct 30 01:23:47.076072 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.075574    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"
Oct 30 01:23:47.076072 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.075597    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" event={"ID":"b92022e8-a46c-45cd-a161-cad0a4af03f0","Type":"ContainerDied","Data":"c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436"}
Oct 30 01:23:47.076072 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.075653    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw" event={"ID":"b92022e8-a46c-45cd-a161-cad0a4af03f0","Type":"ContainerDied","Data":"f577e335e998760f8e4cb951c43e8c2ab191b63db9ad443073e713da3038603c"}
Oct 30 01:23:47.076072 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.075683    6011 scope.go:122] "RemoveContainer" containerID="c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436"
Oct 30 01:23:47.077919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.077738    6011 generic.go:356] "Generic (PLEG): container finished" podID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerID="153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3" exitCode=0
Oct 30 01:23:47.077919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.077792    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" event={"ID":"6edb9b4d-ba8c-4679-8ee4-a36537ac205e","Type":"ContainerDied","Data":"153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3"}
Oct 30 01:23:47.077919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.077832    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"
Oct 30 01:23:47.077919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.077883    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7" event={"ID":"6edb9b4d-ba8c-4679-8ee4-a36537ac205e","Type":"ContainerDied","Data":"67c1f401504f24104ef5a7ad2ed2fb8377425491917d159d25d3b671f36bf735"}
Oct 30 01:23:47.099909 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.099856    6011 scope.go:122] "RemoveContainer" containerID="c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436"
Oct 30 01:23:47.101052 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:47.101016    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436\": container with ID starting with c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436 not found: ID does not exist" containerID="c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436"
Oct 30 01:23:47.101229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.101088    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436"} err="failed to get container status \"c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436\": rpc error: code = NotFound desc = could not find container \"c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436\": container with ID starting with c0481a6e4ad4c42254a2cd28386c41f474b1aff13c60b880c1eece7d95d9c436 not found: ID does not exist"
Oct 30 01:23:47.101229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.101120    6011 scope.go:122] "RemoveContainer" containerID="153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3"
Oct 30 01:23:47.114345 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.114317    6011 scope.go:122] "RemoveContainer" containerID="153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3"
Oct 30 01:23:47.114855 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:47.114827    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3\": container with ID starting with 153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3 not found: ID does not exist" containerID="153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3"
Oct 30 01:23:47.114968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.114867    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3"} err="failed to get container status \"153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3\": rpc error: code = NotFound desc = could not find container \"153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3\": container with ID starting with 153e9713cbea7119e341e2e1aa870c8f7f59db3096852071d5dd952959e012e3 not found: ID does not exist"
Oct 30 01:23:47.158918 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.158883    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"]
Oct 30 01:23:47.171898 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.171863    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-qsjf7"]
Oct 30 01:23:47.197777 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.197738    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"]
Oct 30 01:23:47.201323 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.201294    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-07a3a8-openshift-default-5d699d59b5-474qw"]
Oct 30 01:23:47.838472 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.838072    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:23:47.871864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.871835    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-data\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.871982 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.871886    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51759780-e51e-4310-be1b-f82b6c56f28d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51759780-e51e-4310-be1b-f82b6c56f28d-istiod-ca-cert\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.871987    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-kube-api-access-s94gq\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-kube-api-access-s94gq\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872035    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-socket\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872156 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872127    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/51759780-e51e-4310-be1b-f82b6c56f28d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51759780-e51e-4310-be1b-f82b6c56f28d-istio-podinfo\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872258 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872141    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-data" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.872258 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872226    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-envoy\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872258 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872250    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51759780-e51e-4310-be1b-f82b6c56f28d-istiod-ca-cert" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:47.872424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872266    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-credential-socket\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872298    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-istio-token\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-istio-token\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872329    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-certs\") pod \"51759780-e51e-4310-be1b-f82b6c56f28d\" (UID: \"51759780-e51e-4310-be1b-f82b6c56f28d\") "
Oct 30 01:23:47.872570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872536    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-socket" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.872684 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872660    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-certs" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.872746 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872733    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-credential-socket" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.873005 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.872979    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.873087 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.873004    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/51759780-e51e-4310-be1b-f82b6c56f28d-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.873087 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.873020    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.873087 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.873033    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.873087 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.873048    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.874489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.874463    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/51759780-e51e-4310-be1b-f82b6c56f28d-istio-podinfo" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:23:47.874614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.874562    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-envoy" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.874812 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.874796    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-kube-api-access-s94gq" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "kube-api-access-s94gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:47.875003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.874985    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-istio-token" pod "51759780-e51e-4310-be1b-f82b6c56f28d" (UID: "51759780-e51e-4310-be1b-f82b6c56f28d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:47.920921 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.920870    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:23:47.924358 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.924330    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:23:47.974379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974341    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-certs\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.974597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974397    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-certs\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.974729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974701    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-certs" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.974729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974715    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-certs" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.974867 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974751    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/91adfc76-ab32-46cd-952c-97daab8d00db-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91adfc76-ab32-46cd-952c-97daab8d00db-istio-podinfo\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.974867 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974807    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-podinfo\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.974867 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974855    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-token\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-token\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.975025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974925    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/91adfc76-ab32-46cd-952c-97daab8d00db-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91adfc76-ab32-46cd-952c-97daab8d00db-istiod-ca-cert\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.975025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.974965    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-envoy\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.975129 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975033    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-kube-api-access-2p5bb\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-kube-api-access-2p5bb\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.975129 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975065    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-data\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.975129 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975096    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/40cda466-cc83-4396-9adf-42cf7db07cfe-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cda466-cc83-4396-9adf-42cf7db07cfe-istiod-ca-cert\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.975379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975144    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-istio-token\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-istio-token\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.975379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975218    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-kube-api-access-lxx2g\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-kube-api-access-lxx2g\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.975379 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975316    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-envoy\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.975540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975402    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-data\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.975759 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975667    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-credential-socket\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.975759 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975706    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-credential-socket\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.975759 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975749    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-socket\") pod \"40cda466-cc83-4396-9adf-42cf7db07cfe\" (UID: \"40cda466-cc83-4396-9adf-42cf7db07cfe\") "
Oct 30 01:23:47.975923 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.975813    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-socket\") pod \"91adfc76-ab32-46cd-952c-97daab8d00db\" (UID: \"91adfc76-ab32-46cd-952c-97daab8d00db\") "
Oct 30 01:23:47.976632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.976603    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s94gq\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-kube-api-access-s94gq\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.976727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.976638    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/51759780-e51e-4310-be1b-f82b6c56f28d-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.976727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.976655    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/51759780-e51e-4310-be1b-f82b6c56f28d-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.976727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.976668    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/51759780-e51e-4310-be1b-f82b6c56f28d-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.976727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.976681    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.976727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.976694    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:47.976919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.976897    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-socket" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.977409    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-data" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.977447    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-credential-socket" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.977841    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-credential-socket" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.978352    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-socket" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.978971    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/91adfc76-ab32-46cd-952c-97daab8d00db-istio-podinfo" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.979313    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-podinfo" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.979363    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91adfc76-ab32-46cd-952c-97daab8d00db-istiod-ca-cert" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.979650    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-data" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.980437 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.979944    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40cda466-cc83-4396-9adf-42cf7db07cfe-istiod-ca-cert" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:47.980931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.980629    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-envoy" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.980931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.980691    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-kube-api-access-lxx2g" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "kube-api-access-lxx2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:47.981442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.981415    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-token" pod "40cda466-cc83-4396-9adf-42cf7db07cfe" (UID: "40cda466-cc83-4396-9adf-42cf7db07cfe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:47.982470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.982445    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-istio-token" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:47.982672 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.982651    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-envoy" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:23:47.983407 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:47.983373    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-kube-api-access-2p5bb" pod "91adfc76-ab32-46cd-952c-97daab8d00db" (UID: "91adfc76-ab32-46cd-952c-97daab8d00db"). InnerVolumeSpecName "kube-api-access-2p5bb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:48.025273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.025236    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" path="/var/lib/kubelet/pods/6edb9b4d-ba8c-4679-8ee4-a36537ac205e/volumes"
Oct 30 01:23:48.025795 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.025778    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" path="/var/lib/kubelet/pods/b92022e8-a46c-45cd-a161-cad0a4af03f0/volumes"
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078363    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40cda466-cc83-4396-9adf-42cf7db07cfe-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078397    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078413    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxx2g\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-kube-api-access-lxx2g\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078427    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078443    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078458    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078472    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078487    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40cda466-cc83-4396-9adf-42cf7db07cfe-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078501    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078513    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91adfc76-ab32-46cd-952c-97daab8d00db-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078523    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078531    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40cda466-cc83-4396-9adf-42cf7db07cfe-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078539    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91adfc76-ab32-46cd-952c-97daab8d00db-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078547    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078555    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p5bb\" (UniqueName: \"kubernetes.io/projected/91adfc76-ab32-46cd-952c-97daab8d00db-kube-api-access-2p5bb\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.078597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.078562    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91adfc76-ab32-46cd-952c-97daab8d00db-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:48.089154 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.089048    6011 generic.go:356] "Generic (PLEG): container finished" podID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerID="38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25" exitCode=0
Oct 30 01:23:48.089154 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.089099    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"
Oct 30 01:23:48.089154 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.089134    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" event={"ID":"40cda466-cc83-4396-9adf-42cf7db07cfe","Type":"ContainerDied","Data":"38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25"}
Oct 30 01:23:48.089527 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.089240    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj" event={"ID":"40cda466-cc83-4396-9adf-42cf7db07cfe","Type":"ContainerDied","Data":"9f728e5dfc3ba16b69ec6219f8dbcd760a94d1b67d34b00ea9f6b560c667bddb"}
Oct 30 01:23:48.089527 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.089270    6011 scope.go:122] "RemoveContainer" containerID="38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25"
Oct 30 01:23:48.099495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.099457    6011 generic.go:356] "Generic (PLEG): container finished" podID="51759780-e51e-4310-be1b-f82b6c56f28d" containerID="e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a" exitCode=0
Oct 30 01:23:48.099824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.099605    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"
Oct 30 01:23:48.099941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.099622    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" event={"ID":"51759780-e51e-4310-be1b-f82b6c56f28d","Type":"ContainerDied","Data":"e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a"}
Oct 30 01:23:48.100002 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.099939    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh" event={"ID":"51759780-e51e-4310-be1b-f82b6c56f28d","Type":"ContainerDied","Data":"3d0e3d2dda79ab8a69662dae4c3b47531cc05bed695b832641c29e64e31716da"}
Oct 30 01:23:48.103528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.102814    6011 generic.go:356] "Generic (PLEG): container finished" podID="91adfc76-ab32-46cd-952c-97daab8d00db" containerID="ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03" exitCode=0
Oct 30 01:23:48.103528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.102863    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" event={"ID":"91adfc76-ab32-46cd-952c-97daab8d00db","Type":"ContainerDied","Data":"ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03"}
Oct 30 01:23:48.103528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.102902    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864" event={"ID":"91adfc76-ab32-46cd-952c-97daab8d00db","Type":"ContainerDied","Data":"67486d9ac0a0b5739bb02372a54343c081df6ffb64a5a039cc743f77a954b21c"}
Oct 30 01:23:48.103528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.102957    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"
Oct 30 01:23:48.110186 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.110151    6011 scope.go:122] "RemoveContainer" containerID="38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25"
Oct 30 01:23:48.110631 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:48.110602    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25\": container with ID starting with 38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25 not found: ID does not exist" containerID="38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25"
Oct 30 01:23:48.110752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.110636    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25"} err="failed to get container status \"38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25\": rpc error: code = NotFound desc = could not find container \"38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25\": container with ID starting with 38d6457edf7bff73630db6b6eed623172e51596fee4a6c5d946b1364aeb35b25 not found: ID does not exist"
Oct 30 01:23:48.110752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.110654    6011 scope.go:122] "RemoveContainer" containerID="e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a"
Oct 30 01:23:48.121610 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.121569    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" podUID="cffdfd57-7d46-40f0-89c3-5d74d036b541" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:23:48.124352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.124324    6011 scope.go:122] "RemoveContainer" containerID="e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a"
Oct 30 01:23:48.125766 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:48.125713    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a\": container with ID starting with e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a not found: ID does not exist" containerID="e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a"
Oct 30 01:23:48.125852 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.125769    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a"} err="failed to get container status \"e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a\": rpc error: code = NotFound desc = could not find container \"e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a\": container with ID starting with e916aa4cc5529f23bf2da10d6d981be61e594a66a7fa52f3f920f9f8f886351a not found: ID does not exist"
Oct 30 01:23:48.125852 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.125797    6011 scope.go:122] "RemoveContainer" containerID="ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03"
Oct 30 01:23:48.137019 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.136995    6011 scope.go:122] "RemoveContainer" containerID="ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03"
Oct 30 01:23:48.137435 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:48.137410    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03\": container with ID starting with ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03 not found: ID does not exist" containerID="ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03"
Oct 30 01:23:48.137554 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.137441    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03"} err="failed to get container status \"ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03\": rpc error: code = NotFound desc = could not find container \"ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03\": container with ID starting with ef853e4ba9113d087af8b80a0b883e3be65f748d110a25f1e6ff5fb6b1151d03 not found: ID does not exist"
Oct 30 01:23:48.168238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.168205    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"]
Oct 30 01:23:48.172810 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.172781    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-ntwjj"]
Oct 30 01:23:48.246318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.246275    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"]
Oct 30 01:23:48.251815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.251779    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-hcdqh"]
Oct 30 01:23:48.272495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.271403    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"]
Oct 30 01:23:48.276682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:48.276649    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-5c515e-openshift-default-7c9b898887-b2864"]
Oct 30 01:23:50.023536 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:50.023498    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" path="/var/lib/kubelet/pods/40cda466-cc83-4396-9adf-42cf7db07cfe/volumes"
Oct 30 01:23:50.024051 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:50.024035    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" path="/var/lib/kubelet/pods/51759780-e51e-4310-be1b-f82b6c56f28d/volumes"
Oct 30 01:23:50.024508 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:50.024494    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" path="/var/lib/kubelet/pods/91adfc76-ab32-46cd-952c-97daab8d00db/volumes"
Oct 30 01:23:52.084864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:52.084832    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:23:55.709127 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:55.709085    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"]
Oct 30 01:23:55.709685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:55.709429    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5" podUID="ac84a502-752f-4b57-a13e-020a191f0d09" containerName="maas-api" containerID="cri-o://8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7" gracePeriod=30
Oct 30 01:23:56.065878 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.065748    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:23:56.131246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.127551    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-maas-api-tls\") pod \"ac84a502-752f-4b57-a13e-020a191f0d09\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") "
Oct 30 01:23:56.131246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.127632    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-metrics-certs\") pod \"ac84a502-752f-4b57-a13e-020a191f0d09\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") "
Oct 30 01:23:56.131246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.127744    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac84a502-752f-4b57-a13e-020a191f0d09-kube-api-access-7dlt6\" (UniqueName: \"kubernetes.io/projected/ac84a502-752f-4b57-a13e-020a191f0d09-kube-api-access-7dlt6\") pod \"ac84a502-752f-4b57-a13e-020a191f0d09\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") "
Oct 30 01:23:56.131246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.128021    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac84a502-752f-4b57-a13e-020a191f0d09-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac84a502-752f-4b57-a13e-020a191f0d09-openshift-service-ca\") pod \"ac84a502-752f-4b57-a13e-020a191f0d09\" (UID: \"ac84a502-752f-4b57-a13e-020a191f0d09\") "
Oct 30 01:23:56.131246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.128472    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac84a502-752f-4b57-a13e-020a191f0d09-openshift-service-ca" pod "ac84a502-752f-4b57-a13e-020a191f0d09" (UID: "ac84a502-752f-4b57-a13e-020a191f0d09"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:23:56.131246 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.128914    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac84a502-752f-4b57-a13e-020a191f0d09-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:56.133212 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.133060    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac84a502-752f-4b57-a13e-020a191f0d09-kube-api-access-7dlt6" pod "ac84a502-752f-4b57-a13e-020a191f0d09" (UID: "ac84a502-752f-4b57-a13e-020a191f0d09"). InnerVolumeSpecName "kube-api-access-7dlt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:23:56.134014 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.133992    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-metrics-certs" pod "ac84a502-752f-4b57-a13e-020a191f0d09" (UID: "ac84a502-752f-4b57-a13e-020a191f0d09"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:56.135817 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.135788    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-maas-api-tls" pod "ac84a502-752f-4b57-a13e-020a191f0d09" (UID: "ac84a502-752f-4b57-a13e-020a191f0d09"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:23:56.173067 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.173032    6011 generic.go:356] "Generic (PLEG): container finished" podID="ac84a502-752f-4b57-a13e-020a191f0d09" containerID="8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7" exitCode=0
Oct 30 01:23:56.173294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.173093    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"
Oct 30 01:23:56.173294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.173121    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5" event={"ID":"ac84a502-752f-4b57-a13e-020a191f0d09","Type":"ContainerDied","Data":"8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7"}
Oct 30 01:23:56.173294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.173211    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5" event={"ID":"ac84a502-752f-4b57-a13e-020a191f0d09","Type":"ContainerDied","Data":"bd4f24ffcafc206a744db8809bf338bcee7b884666b7524147a2b31078052980"}
Oct 30 01:23:56.173294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.173235    6011 scope.go:122] "RemoveContainer" containerID="8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7"
Oct 30 01:23:56.186091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.186067    6011 scope.go:122] "RemoveContainer" containerID="8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7"
Oct 30 01:23:56.186470 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:56.186446    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7\": container with ID starting with 8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7 not found: ID does not exist" containerID="8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7"
Oct 30 01:23:56.186567 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.186477    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7"} err="failed to get container status \"8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7\": rpc error: code = NotFound desc = could not find container \"8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7\": container with ID starting with 8ccdc4432910c26de7a9a69f55a8d3980b14ee73b1c0d79613efddec9a38d9e7 not found: ID does not exist"
Oct 30 01:23:56.233697 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.233642    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"]
Oct 30 01:23:56.234628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.234599    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:56.234628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.234630    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac84a502-752f-4b57-a13e-020a191f0d09-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:56.234828 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.234645    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dlt6\" (UniqueName: \"kubernetes.io/projected/ac84a502-752f-4b57-a13e-020a191f0d09-kube-api-access-7dlt6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:23:56.241398 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:56.241357    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-96bd13-769fd84d5f-dbfw5"]
Oct 30 01:23:57.912279 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:23:57.912236    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:23:58.022780 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:58.022745    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac84a502-752f-4b57-a13e-020a191f0d09" path="/var/lib/kubelet/pods/ac84a502-752f-4b57-a13e-020a191f0d09/volumes"
Oct 30 01:23:58.122897 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:23:58.122846    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" podUID="cffdfd57-7d46-40f0-89c3-5d74d036b541" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:24:00.730211 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:00.730013    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"]
Oct 30 01:24:00.731230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:00.730311    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy" containerID="cri-o://f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd" gracePeriod=30
Oct 30 01:24:01.817916 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:01.817886    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:24:01.913425 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:01.913392    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cffdfd57-7d46-40f0-89c3-5d74d036b541-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cffdfd57-7d46-40f0-89c3-5d74d036b541-plugins-config-volume\") pod \"cffdfd57-7d46-40f0-89c3-5d74d036b541\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") "
Oct 30 01:24:01.913628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:01.913470    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cffdfd57-7d46-40f0-89c3-5d74d036b541-kube-api-access-8vs4w\" (UniqueName: \"kubernetes.io/projected/cffdfd57-7d46-40f0-89c3-5d74d036b541-kube-api-access-8vs4w\") pod \"cffdfd57-7d46-40f0-89c3-5d74d036b541\" (UID: \"cffdfd57-7d46-40f0-89c3-5d74d036b541\") "
Oct 30 01:24:01.913837 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:01.913810    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cffdfd57-7d46-40f0-89c3-5d74d036b541-plugins-config-volume" pod "cffdfd57-7d46-40f0-89c3-5d74d036b541" (UID: "cffdfd57-7d46-40f0-89c3-5d74d036b541"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:01.914228 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:01.914209    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cffdfd57-7d46-40f0-89c3-5d74d036b541-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:01.915922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:01.915900    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cffdfd57-7d46-40f0-89c3-5d74d036b541-kube-api-access-8vs4w" pod "cffdfd57-7d46-40f0-89c3-5d74d036b541" (UID: "cffdfd57-7d46-40f0-89c3-5d74d036b541"). InnerVolumeSpecName "kube-api-access-8vs4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:02.015458 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.015414    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8vs4w\" (UniqueName: \"kubernetes.io/projected/cffdfd57-7d46-40f0-89c3-5d74d036b541-kube-api-access-8vs4w\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:02.213647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.213598    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"]
Oct 30 01:24:02.214352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.214312    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm" podUID="71747683-1ef4-4e14-99a1-7c8d7cea3e5e" containerName="maas-api" containerID="cri-o://f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb" gracePeriod=30
Oct 30 01:24:02.265134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.265101    6011 generic.go:356] "Generic (PLEG): container finished" podID="cffdfd57-7d46-40f0-89c3-5d74d036b541" containerID="85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b" exitCode=0
Oct 30 01:24:02.265365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.265168    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" event={"ID":"cffdfd57-7d46-40f0-89c3-5d74d036b541","Type":"ContainerDied","Data":"85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b"}
Oct 30 01:24:02.265365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.265240    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2" event={"ID":"cffdfd57-7d46-40f0-89c3-5d74d036b541","Type":"ContainerDied","Data":"8aebbd8d1ab4a951f0011ea8c08b9ff72040c76fa1bce9bce6540f1c44de26fa"}
Oct 30 01:24:02.265365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.265268    6011 scope.go:122] "RemoveContainer" containerID="85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b"
Oct 30 01:24:02.265365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.265267    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"
Oct 30 01:24:02.324470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.324398    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"]
Oct 30 01:24:02.327904 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.327870    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-24d5ce-d87b5465d-dzlz2"]
Oct 30 01:24:02.330618 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.330595    6011 scope.go:122] "RemoveContainer" containerID="85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b"
Oct 30 01:24:02.330954 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:02.330934    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b\": container with ID starting with 85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b not found: ID does not exist" containerID="85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b"
Oct 30 01:24:02.331002 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.330963    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b"} err="failed to get container status \"85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b\": rpc error: code = NotFound desc = could not find container \"85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b\": container with ID starting with 85c4293ed3bc5c3705a78322843b2466e7bc85928842cb46fe4a8bd61bbb490b not found: ID does not exist"
Oct 30 01:24:02.556687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.556222    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 01:24:02.626832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.626736    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-kube-api-access-zzkj6\" (UniqueName: \"kubernetes.io/projected/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-kube-api-access-zzkj6\") pod \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") "
Oct 30 01:24:02.627009 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.626902    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-metrics-certs\") pod \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") "
Oct 30 01:24:02.627073 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.627004    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-openshift-service-ca\") pod \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") "
Oct 30 01:24:02.627129 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.627108    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-maas-api-tls\") pod \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\" (UID: \"71747683-1ef4-4e14-99a1-7c8d7cea3e5e\") "
Oct 30 01:24:02.627605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.627571    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-openshift-service-ca" pod "71747683-1ef4-4e14-99a1-7c8d7cea3e5e" (UID: "71747683-1ef4-4e14-99a1-7c8d7cea3e5e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:02.627824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.627760    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:02.629627 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.629592    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-metrics-certs" pod "71747683-1ef4-4e14-99a1-7c8d7cea3e5e" (UID: "71747683-1ef4-4e14-99a1-7c8d7cea3e5e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:02.631390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.631353    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-maas-api-tls" pod "71747683-1ef4-4e14-99a1-7c8d7cea3e5e" (UID: "71747683-1ef4-4e14-99a1-7c8d7cea3e5e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:02.631726 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.631700    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-kube-api-access-zzkj6" pod "71747683-1ef4-4e14-99a1-7c8d7cea3e5e" (UID: "71747683-1ef4-4e14-99a1-7c8d7cea3e5e"). InnerVolumeSpecName "kube-api-access-zzkj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:02.728857 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.728820    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:02.728857 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.728853    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzkj6\" (UniqueName: \"kubernetes.io/projected/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-kube-api-access-zzkj6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:02.728857 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:02.728864    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/71747683-1ef4-4e14-99a1-7c8d7cea3e5e-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:03.275098 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.275065    6011 generic.go:356] "Generic (PLEG): container finished" podID="71747683-1ef4-4e14-99a1-7c8d7cea3e5e" containerID="f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb" exitCode=0
Oct 30 01:24:03.275637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.275117    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"
Oct 30 01:24:03.275637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.275146    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm" event={"ID":"71747683-1ef4-4e14-99a1-7c8d7cea3e5e","Type":"ContainerDied","Data":"f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb"}
Oct 30 01:24:03.275637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.275233    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm" event={"ID":"71747683-1ef4-4e14-99a1-7c8d7cea3e5e","Type":"ContainerDied","Data":"1f93b02bb5f215daca7a17e3fc333f8e3aa9d2a02021a5b8ddec1de287fe33da"}
Oct 30 01:24:03.275637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.275258    6011 scope.go:122] "RemoveContainer" containerID="f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb"
Oct 30 01:24:03.288422 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.288397    6011 scope.go:122] "RemoveContainer" containerID="f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb"
Oct 30 01:24:03.288821 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:03.288793    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb\": container with ID starting with f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb not found: ID does not exist" containerID="f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb"
Oct 30 01:24:03.288913 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.288833    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb"} err="failed to get container status \"f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb\": rpc error: code = NotFound desc = could not find container \"f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb\": container with ID starting with f79d1eeac955154238a8ae50ebc11d78a8d7ba6489281a387394f9d2a169a8bb not found: ID does not exist"
Oct 30 01:24:03.333758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.333724    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"]
Oct 30 01:24:03.369215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:03.363456    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-3c2596-599bb8d7d4-dklmm"]
Oct 30 01:24:04.037226 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:04.036910    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71747683-1ef4-4e14-99a1-7c8d7cea3e5e" path="/var/lib/kubelet/pods/71747683-1ef4-4e14-99a1-7c8d7cea3e5e/volumes"
Oct 30 01:24:04.037564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:04.037540    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cffdfd57-7d46-40f0-89c3-5d74d036b541" path="/var/lib/kubelet/pods/cffdfd57-7d46-40f0-89c3-5d74d036b541/volumes"
Oct 30 01:24:06.037699 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.037667    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:24:06.103324 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103284    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-credential-socket\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103361    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-podinfo\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103458    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-data\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103580    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-socket\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103631    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-envoy\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103766 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103674    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-certs\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103766 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103742    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istiod-ca-cert\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103869 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103824    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-token\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-token\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.103926 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.103914    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-kube-api-access-c6l5k\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-kube-api-access-c6l5k\") pod \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\" (UID: \"138f9a89-b2d0-48ef-80a5-2517d6074e8b\") "
Oct 30 01:24:06.104273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.104245    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-socket" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:06.104830 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.104805    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-data" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:06.105135 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.105028    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-certs" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:06.105290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.104950    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.107275 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.105548    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-credential-socket" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:06.107275 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.105639    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istiod-ca-cert" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:06.110219 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.108392    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-podinfo" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:24:06.110219 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.108514    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-kube-api-access-c6l5k" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "kube-api-access-c6l5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:06.110219 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.108555    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-token" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:06.110219 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.109318    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-envoy" pod "138f9a89-b2d0-48ef-80a5-2517d6074e8b" (UID: "138f9a89-b2d0-48ef-80a5-2517d6074e8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:06.210365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210326    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.210365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210363    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.210660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210377    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.210660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210391    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.210660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210403    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.210660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210417    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6l5k\" (UniqueName: \"kubernetes.io/projected/138f9a89-b2d0-48ef-80a5-2517d6074e8b-kube-api-access-c6l5k\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.210660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210432    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/138f9a89-b2d0-48ef-80a5-2517d6074e8b-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.210660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.210445    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/138f9a89-b2d0-48ef-80a5-2517d6074e8b-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:06.224384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.224346    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"]
Oct 30 01:24:06.224661 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.224628    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy" containerID="cri-o://8e6447c6c21853009a5245a6c9d0097a02d6ca3d2c79998dfde7040bf0f30819" gracePeriod=30
Oct 30 01:24:06.379824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.379657    6011 generic.go:356] "Generic (PLEG): container finished" podID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerID="f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd" exitCode=0
Oct 30 01:24:06.379824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.379753    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" event={"ID":"138f9a89-b2d0-48ef-80a5-2517d6074e8b","Type":"ContainerDied","Data":"f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd"}
Oct 30 01:24:06.379824 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.379808    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5" event={"ID":"138f9a89-b2d0-48ef-80a5-2517d6074e8b","Type":"ContainerDied","Data":"26c164f22eb5a89023ac61c4436834d035269c094643fe71bcdbb8a98636cb8f"}
Oct 30 01:24:06.380126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.379839    6011 scope.go:122] "RemoveContainer" containerID="f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd"
Oct 30 01:24:06.380126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.380012    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"
Oct 30 01:24:06.426470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.426340    6011 scope.go:122] "RemoveContainer" containerID="f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd"
Oct 30 01:24:06.434286 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:06.426927    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd\": container with ID starting with f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd not found: ID does not exist" containerID="f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd"
Oct 30 01:24:06.434286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.426969    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd"} err="failed to get container status \"f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd\": rpc error: code = NotFound desc = could not find container \"f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd\": container with ID starting with f5f2e3828c7326c5d4b60a51e9352e4ac558370b2f2be92727f681b4a6083ddd not found: ID does not exist"
Oct 30 01:24:06.451042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.451005    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"]
Oct 30 01:24:06.456299 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:06.456267    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w6-42d56b-openshift-default-658c44cc97-kw2v5"]
Oct 30 01:24:07.818783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:07.818744    6011 patch_prober.go:29] interesting pod/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 01:24:07.819230 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:07.818814    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 01:24:08.024187 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:08.024141    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" path="/var/lib/kubelet/pods/138f9a89-b2d0-48ef-80a5-2517d6074e8b/volumes"
Oct 30 01:24:09.567720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:09.567680    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"]
Oct 30 01:24:09.568285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:09.567961    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy" containerID="cri-o://aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201" gracePeriod=30
Oct 30 01:24:11.428925 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.428879    6011 generic.go:356] "Generic (PLEG): container finished" podID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerID="8e6447c6c21853009a5245a6c9d0097a02d6ca3d2c79998dfde7040bf0f30819" exitCode=0
Oct 30 01:24:11.429516 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.429056    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" event={"ID":"0961b59d-9703-4409-b5ae-c099ca05f66e","Type":"ContainerDied","Data":"8e6447c6c21853009a5245a6c9d0097a02d6ca3d2c79998dfde7040bf0f30819"}
Oct 30 01:24:11.501247 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.501139    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:24:11.630562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630530    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-kube-api-access-zs2nw\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-kube-api-access-zs2nw\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.630742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630603    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-socket\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.630742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630649    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-certs\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.630742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630697    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-credential-socket\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.630742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630725    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0961b59d-9703-4409-b5ae-c099ca05f66e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0961b59d-9703-4409-b5ae-c099ca05f66e-istiod-ca-cert\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.630919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630758    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-data\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.630919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630903    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-envoy\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.631029 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.630962    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-socket" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:11.631084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631055    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-token\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-token\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.631084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631056    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-credential-socket" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:11.631225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631103    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-podinfo\") pod \"0961b59d-9703-4409-b5ae-c099ca05f66e\" (UID: \"0961b59d-9703-4409-b5ae-c099ca05f66e\") "
Oct 30 01:24:11.631225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631133    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-certs" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:11.631343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631222    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0961b59d-9703-4409-b5ae-c099ca05f66e-istiod-ca-cert" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:11.631458 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631394    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-data" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:11.631975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631947    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.631975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631977    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0961b59d-9703-4409-b5ae-c099ca05f66e-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.632160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.631992    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.632160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.632005    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.632160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.632019    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.633294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.633266    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-kube-api-access-zs2nw" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "kube-api-access-zs2nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:11.633500 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.633478    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-token" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:11.633598 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.633561    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-podinfo" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:24:11.633598 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.633570    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-envoy" pod "0961b59d-9703-4409-b5ae-c099ca05f66e" (UID: "0961b59d-9703-4409-b5ae-c099ca05f66e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:11.732856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.732815    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.732856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.732851    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.732856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.732860    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0961b59d-9703-4409-b5ae-c099ca05f66e-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:11.733102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:11.732869    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs2nw\" (UniqueName: \"kubernetes.io/projected/0961b59d-9703-4409-b5ae-c099ca05f66e-kube-api-access-zs2nw\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:12.437578 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:12.437546    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"
Oct 30 01:24:12.438015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:12.437544    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp" event={"ID":"0961b59d-9703-4409-b5ae-c099ca05f66e","Type":"ContainerDied","Data":"91e78b0ba004229446190f3a9fa5faf600ed0719435d3a8c166aee0ef3229e92"}
Oct 30 01:24:12.438015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:12.437681    6011 scope.go:122] "RemoveContainer" containerID="8e6447c6c21853009a5245a6c9d0097a02d6ca3d2c79998dfde7040bf0f30819"
Oct 30 01:24:12.507039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:12.507000    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"]
Oct 30 01:24:12.520484 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:12.520451    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-24d5ce-openshift-default-55fb54db69-mf8dp"]
Oct 30 01:24:14.022409 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.022367    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" path="/var/lib/kubelet/pods/0961b59d-9703-4409-b5ae-c099ca05f66e/volumes"
Oct 30 01:24:14.854469 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.854404    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 01:24:14.875338 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875304    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-credential-socket\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875377    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/caf03273-5153-4145-907a-1495fadd7036-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf03273-5153-4145-907a-1495fadd7036-istiod-ca-cert\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875414    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-envoy\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875442    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-socket\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875602    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/caf03273-5153-4145-907a-1495fadd7036-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf03273-5153-4145-907a-1495fadd7036-istio-podinfo\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875643    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-certs\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875711    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-kube-api-access-jhlzc\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-kube-api-access-jhlzc\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875786    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-data\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.875886 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.875854    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-istio-token\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-istio-token\") pod \"caf03273-5153-4145-907a-1495fadd7036\" (UID: \"caf03273-5153-4145-907a-1495fadd7036\") "
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876082    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-credential-socket" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876248    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-socket" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876288    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-certs" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876514    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-data" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876558    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caf03273-5153-4145-907a-1495fadd7036-istiod-ca-cert" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876542    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876612    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.876834 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.876742    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.879839 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.879811    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-envoy" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:14.880186 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.880154    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-kube-api-access-jhlzc" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "kube-api-access-jhlzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:14.880283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.880202    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-istio-token" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:14.905239 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.905137    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/caf03273-5153-4145-907a-1495fadd7036-istio-podinfo" pod "caf03273-5153-4145-907a-1495fadd7036" (UID: "caf03273-5153-4145-907a-1495fadd7036"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:24:14.979502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.979465    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/caf03273-5153-4145-907a-1495fadd7036-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.979502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.979498    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhlzc\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-kube-api-access-jhlzc\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.979502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.979508    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.979836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.979517    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/caf03273-5153-4145-907a-1495fadd7036-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.979836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.979526    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/caf03273-5153-4145-907a-1495fadd7036-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:14.979836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:14.979534    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/caf03273-5153-4145-907a-1495fadd7036-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:15.288911 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.288872    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"]
Oct 30 01:24:15.289450 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.289095    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg" podUID="dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" containerName="maas-api" containerID="cri-o://6d1cb9b2e8e2e0a621075964ea541d389536a6ee485cb0609a8a4b3d1c1b195d" gracePeriod=30
Oct 30 01:24:15.482322 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.482284    6011 generic.go:356] "Generic (PLEG): container finished" podID="dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" containerID="6d1cb9b2e8e2e0a621075964ea541d389536a6ee485cb0609a8a4b3d1c1b195d" exitCode=0
Oct 30 01:24:15.482503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.482357    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg" event={"ID":"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426","Type":"ContainerDied","Data":"6d1cb9b2e8e2e0a621075964ea541d389536a6ee485cb0609a8a4b3d1c1b195d"}
Oct 30 01:24:15.484503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.484478    6011 generic.go:356] "Generic (PLEG): container finished" podID="caf03273-5153-4145-907a-1495fadd7036" containerID="aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201" exitCode=0
Oct 30 01:24:15.484632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.484554    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"
Oct 30 01:24:15.484632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.484565    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" event={"ID":"caf03273-5153-4145-907a-1495fadd7036","Type":"ContainerDied","Data":"aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201"}
Oct 30 01:24:15.484632 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.484622    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x" event={"ID":"caf03273-5153-4145-907a-1495fadd7036","Type":"ContainerDied","Data":"bf157d97515a5392e965cc03355164d032f336f9958a4b57f7fd6ce9f6c5322f"}
Oct 30 01:24:15.484774 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.484653    6011 scope.go:122] "RemoveContainer" containerID="aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201"
Oct 30 01:24:15.498793 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.498769    6011 scope.go:122] "RemoveContainer" containerID="aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201"
Oct 30 01:24:15.499200 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:15.499139    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201\": container with ID starting with aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201 not found: ID does not exist" containerID="aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201"
Oct 30 01:24:15.499286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.499195    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201"} err="failed to get container status \"aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201\": rpc error: code = NotFound desc = could not find container \"aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201\": container with ID starting with aa12a7aa8d6a3c6ac28e5ad2b6ed40b89817a20d1c730e26db3fda5a6c15b201 not found: ID does not exist"
Oct 30 01:24:15.563261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.554914    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"]
Oct 30 01:24:15.571322 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.568856    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-3c2596-openshift-default-6b68bd79df-h8z5x"]
Oct 30 01:24:15.713116 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.712831    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:24:15.791524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.791491    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-openshift-service-ca\") pod \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") "
Oct 30 01:24:15.791736 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.791574    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-metrics-certs\") pod \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") "
Oct 30 01:24:15.791736 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.791653    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-kube-api-access-jd5w6\" (UniqueName: \"kubernetes.io/projected/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-kube-api-access-jd5w6\") pod \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") "
Oct 30 01:24:15.791848 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.791830    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-maas-api-tls\") pod \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\" (UID: \"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426\") "
Oct 30 01:24:15.791980 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.791953    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-openshift-service-ca" pod "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" (UID: "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:15.792395 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.792370    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:15.794491 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.794468    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-metrics-certs" pod "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" (UID: "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:15.794617 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.794497    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-kube-api-access-jd5w6" pod "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" (UID: "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426"). InnerVolumeSpecName "kube-api-access-jd5w6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:15.798277 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.798255    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-maas-api-tls" pod "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" (UID: "dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:15.894024 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.893921    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:15.894024 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.893958    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jd5w6\" (UniqueName: \"kubernetes.io/projected/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-kube-api-access-jd5w6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:15.894024 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:15.893973    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:16.022105 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:16.022060    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="caf03273-5153-4145-907a-1495fadd7036" path="/var/lib/kubelet/pods/caf03273-5153-4145-907a-1495fadd7036/volumes"
Oct 30 01:24:16.497326 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:16.497295    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"
Oct 30 01:24:16.497859 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:16.497335    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg" event={"ID":"dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426","Type":"ContainerDied","Data":"9510e561ec7054720b531a2218b91f6751efa455990d32fd86969e90f9238a9a"}
Oct 30 01:24:16.497859 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:16.497403    6011 scope.go:122] "RemoveContainer" containerID="6d1cb9b2e8e2e0a621075964ea541d389536a6ee485cb0609a8a4b3d1c1b195d"
Oct 30 01:24:16.552630 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:16.552597    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"]
Oct 30 01:24:16.555575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:16.555547    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w5-b05110-84684b79fc-vdhtg"]
Oct 30 01:24:18.028704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:18.028671    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" path="/var/lib/kubelet/pods/dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426/volumes"
Oct 30 01:24:25.925897 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:25.925851    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"]
Oct 30 01:24:25.926491 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:25.926128    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy" containerID="cri-o://6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7" gracePeriod=30
Oct 30 01:24:27.912117 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:27.912032    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:24:31.200614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.200546    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:24:31.347539 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347492    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-envoy\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.347742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347555    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-credential-socket\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.347742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347610    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-data\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.347742 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347675    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-token\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-token\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.347914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347758    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-socket\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.347914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347798    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-certs\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.347914 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347896    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-kube-api-access-qfch9\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-kube-api-access-qfch9\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.348056 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347976    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-data" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:31.348056 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.347991    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istiod-ca-cert\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.348157 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.348099    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-podinfo\") pod \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\" (UID: \"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc\") "
Oct 30 01:24:31.348256 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.348224    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-certs" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:31.348587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.348564    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-credential-socket" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:31.348774 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.348671    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.348894 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.348879    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.349252 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.349232    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istiod-ca-cert" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:31.350371 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.349640    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-socket" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:31.351383 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.351294    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-kube-api-access-qfch9" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "kube-api-access-qfch9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:31.351383 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.351326    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-token" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:31.351518 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.351412    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-envoy" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:24:31.351666 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.351643    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-podinfo" pod "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" (UID: "d1aae6be-9828-4b54-8c2e-039c6bb9dfbc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:24:31.450703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.450606    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.450703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.450645    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.450703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.450655    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.450703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.450664    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.450703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.450674    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.450703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.450684    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.450703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.450696    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfch9\" (UniqueName: \"kubernetes.io/projected/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc-kube-api-access-qfch9\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:31.699070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.699031    6011 generic.go:356] "Generic (PLEG): container finished" podID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerID="6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7" exitCode=0
Oct 30 01:24:31.699287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.699112    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"
Oct 30 01:24:31.699287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.699115    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" event={"ID":"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc","Type":"ContainerDied","Data":"6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7"}
Oct 30 01:24:31.699287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.699277    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw" event={"ID":"d1aae6be-9828-4b54-8c2e-039c6bb9dfbc","Type":"ContainerDied","Data":"fb59fbb678710a2915363d067dd8fef2563b94b4018a1c1262d5f144f7dabe00"}
Oct 30 01:24:31.699431 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.699300    6011 scope.go:122] "RemoveContainer" containerID="6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7"
Oct 30 01:24:31.711125 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.711103    6011 scope.go:122] "RemoveContainer" containerID="6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7"
Oct 30 01:24:31.711477 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:31.711455    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7\": container with ID starting with 6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7 not found: ID does not exist" containerID="6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7"
Oct 30 01:24:31.711560 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.711487    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7"} err="failed to get container status \"6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7\": rpc error: code = NotFound desc = could not find container \"6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7\": container with ID starting with 6aad95afea6ce33703cd68cacd9a17685b81842eb17494532879b9fb916132f7 not found: ID does not exist"
Oct 30 01:24:31.755168 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.755124    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"]
Oct 30 01:24:31.768451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:31.768423    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ca324d-openshift-default-788bf89f9f-nt9hw"]
Oct 30 01:24:32.022205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.022099    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" path="/var/lib/kubelet/pods/d1aae6be-9828-4b54-8c2e-039c6bb9dfbc/volumes"
Oct 30 01:24:32.926960 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.926921    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"]
Oct 30 01:24:32.928014 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.927952    6011 cpu_manager.go:414] "containerMap: removing container" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy"
Oct 30 01:24:32.928014 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.927979    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy"
Oct 30 01:24:32.928014 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928001    6011 cpu_manager.go:414] "containerMap: removing container" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy"
Oct 30 01:24:32.928014 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928009    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928042    6011 cpu_manager.go:414] "containerMap: removing container" podUID="ac84a502-752f-4b57-a13e-020a191f0d09" containerName="maas-api"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928050    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac84a502-752f-4b57-a13e-020a191f0d09" containerName="maas-api"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928077    6011 cpu_manager.go:414] "containerMap: removing container" podUID="cffdfd57-7d46-40f0-89c3-5d74d036b541" containerName="payload-processing"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928083    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cffdfd57-7d46-40f0-89c3-5d74d036b541" containerName="payload-processing"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928095    6011 cpu_manager.go:414] "containerMap: removing container" podUID="dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" containerName="maas-api"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928103    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" containerName="maas-api"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928114    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928121    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928140    6011 cpu_manager.go:414] "containerMap: removing container" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928147    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928300    6011 cpu_manager.go:414] "containerMap: removing container" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928313    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928329    6011 cpu_manager.go:414] "containerMap: removing container" podUID="71747683-1ef4-4e14-99a1-7c8d7cea3e5e" containerName="maas-api"
Oct 30 01:24:32.928334 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928336    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71747683-1ef4-4e14-99a1-7c8d7cea3e5e" containerName="maas-api"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928363    6011 cpu_manager.go:414] "containerMap: removing container" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928371    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928380    6011 cpu_manager.go:414] "containerMap: removing container" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928387    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928431    6011 cpu_manager.go:414] "containerMap: removing container" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928438    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928451    6011 cpu_manager.go:414] "containerMap: removing container" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928457    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928644    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac84a502-752f-4b57-a13e-020a191f0d09" containerName="maas-api"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928662    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="cffdfd57-7d46-40f0-89c3-5d74d036b541" containerName="payload-processing"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928673    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc8d0fc6-ba55-4d6f-bb3b-c31246d1d426" containerName="maas-api"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928681    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1aae6be-9828-4b54-8c2e-039c6bb9dfbc" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928691    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="40cda466-cc83-4396-9adf-42cf7db07cfe" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928701    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="51759780-e51e-4310-be1b-f82b6c56f28d" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928710    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="71747683-1ef4-4e14-99a1-7c8d7cea3e5e" containerName="maas-api"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928719    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="caf03273-5153-4145-907a-1495fadd7036" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928728    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="6edb9b4d-ba8c-4679-8ee4-a36537ac205e" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928737    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="b92022e8-a46c-45cd-a161-cad0a4af03f0" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928747    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="0961b59d-9703-4409-b5ae-c099ca05f66e" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928756    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="138f9a89-b2d0-48ef-80a5-2517d6074e8b" containerName="istio-proxy"
Oct 30 01:24:32.929036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.928764    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="91adfc76-ab32-46cd-952c-97daab8d00db" containerName="istio-proxy"
Oct 30 01:24:32.934516 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.934486    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:32.948353 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.948317    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"]
Oct 30 01:24:32.974239 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:32.958233    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-0519f6-serving-cert\""
Oct 30 01:24:33.077287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.077247    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p46ff\" (UniqueName: \"kubernetes.io/projected/48b1a7c2-03cc-4809-ad16-7ef76db065fd-kube-api-access-p46ff\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.077287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.077297    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.077581 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.077437    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-metrics-certs\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.077581 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.077513    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b1a7c2-03cc-4809-ad16-7ef76db065fd-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.182136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.180079    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p46ff\" (UniqueName: \"kubernetes.io/projected/48b1a7c2-03cc-4809-ad16-7ef76db065fd-kube-api-access-p46ff\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.182136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.180160    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.182136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.180257    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-metrics-certs\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.182136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.180323    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b1a7c2-03cc-4809-ad16-7ef76db065fd-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.182136 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.181386    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b1a7c2-03cc-4809-ad16-7ef76db065fd-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.206195 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.206107    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-maas-api-tls\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.207099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.207040    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-metrics-certs\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.214305 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.214236    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p46ff\" (UniqueName: \"kubernetes.io/projected/48b1a7c2-03cc-4809-ad16-7ef76db065fd-kube-api-access-p46ff\") pod \"maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.264814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.264768    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:33.588227 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.588190    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"]
Oct 30 01:24:33.588926 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:24:33.588895    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b1a7c2_03cc_4809_ad16_7ef76db065fd.slice/crio-999aaa5ab90e24a39cd5721ed63042e1caac488d0e2b16b74380dbbeaeacde0d WatchSource:0}: Error finding container 999aaa5ab90e24a39cd5721ed63042e1caac488d0e2b16b74380dbbeaeacde0d: Status 404 returned error can't find the container with id 999aaa5ab90e24a39cd5721ed63042e1caac488d0e2b16b74380dbbeaeacde0d
Oct 30 01:24:33.718395 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:33.718357    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4" event={"ID":"48b1a7c2-03cc-4809-ad16-7ef76db065fd","Type":"ContainerStarted","Data":"999aaa5ab90e24a39cd5721ed63042e1caac488d0e2b16b74380dbbeaeacde0d"}
Oct 30 01:24:35.739750 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:35.739705    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4" event={"ID":"48b1a7c2-03cc-4809-ad16-7ef76db065fd","Type":"ContainerStarted","Data":"1d73ee4d7e8f17a6092a5a2b486ea9f7b4ca6eacdcb152956a6ac59901618309"}
Oct 30 01:24:35.740214 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:35.739891    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:35.780875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:35.780817    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4" podStartSLOduration=2.712815507 podStartE2EDuration="3.780801232s" totalImagesPullingTime="1.067985725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:24:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:24:35.7797626 +0000 UTC m=+2717.882516570" watchObservedRunningTime="2026-10-30 01:24:35.780801232 +0000 UTC m=+2717.883555203"
Oct 30 01:24:38.142076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.141882    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"]
Oct 30 01:24:38.142666 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.142137    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6" podUID="ee8064a2-6df2-402b-a87b-f10caa63c39a" containerName="maas-api" containerID="cri-o://c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28" gracePeriod=30
Oct 30 01:24:38.413924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.413849    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:24:38.561335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.561290    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ee8064a2-6df2-402b-a87b-f10caa63c39a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8064a2-6df2-402b-a87b-f10caa63c39a-openshift-service-ca\") pod \"ee8064a2-6df2-402b-a87b-f10caa63c39a\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") "
Oct 30 01:24:38.561549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.561444    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-metrics-certs\") pod \"ee8064a2-6df2-402b-a87b-f10caa63c39a\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") "
Oct 30 01:24:38.561549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.561490    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls\") pod \"ee8064a2-6df2-402b-a87b-f10caa63c39a\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") "
Oct 30 01:24:38.561655 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.561607    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee8064a2-6df2-402b-a87b-f10caa63c39a-kube-api-access-mg2sz\" (UniqueName: \"kubernetes.io/projected/ee8064a2-6df2-402b-a87b-f10caa63c39a-kube-api-access-mg2sz\") pod \"ee8064a2-6df2-402b-a87b-f10caa63c39a\" (UID: \"ee8064a2-6df2-402b-a87b-f10caa63c39a\") "
Oct 30 01:24:38.561740 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.561707    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee8064a2-6df2-402b-a87b-f10caa63c39a-openshift-service-ca" pod "ee8064a2-6df2-402b-a87b-f10caa63c39a" (UID: "ee8064a2-6df2-402b-a87b-f10caa63c39a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:38.562265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.562240    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ee8064a2-6df2-402b-a87b-f10caa63c39a-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:38.564568 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.564536    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls" pod "ee8064a2-6df2-402b-a87b-f10caa63c39a" (UID: "ee8064a2-6df2-402b-a87b-f10caa63c39a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:38.564680 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.564562    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-metrics-certs" pod "ee8064a2-6df2-402b-a87b-f10caa63c39a" (UID: "ee8064a2-6df2-402b-a87b-f10caa63c39a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:38.564680 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.564612    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8064a2-6df2-402b-a87b-f10caa63c39a-kube-api-access-mg2sz" pod "ee8064a2-6df2-402b-a87b-f10caa63c39a" (UID: "ee8064a2-6df2-402b-a87b-f10caa63c39a"). InnerVolumeSpecName "kube-api-access-mg2sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:38.663490 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.663439    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:38.663490 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.663486    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ee8064a2-6df2-402b-a87b-f10caa63c39a-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:38.663764 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.663501    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg2sz\" (UniqueName: \"kubernetes.io/projected/ee8064a2-6df2-402b-a87b-f10caa63c39a-kube-api-access-mg2sz\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:38.780323 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.780218    6011 generic.go:356] "Generic (PLEG): container finished" podID="ee8064a2-6df2-402b-a87b-f10caa63c39a" containerID="c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28" exitCode=0
Oct 30 01:24:38.780323 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.780312    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"
Oct 30 01:24:38.780556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.780354    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6" event={"ID":"ee8064a2-6df2-402b-a87b-f10caa63c39a","Type":"ContainerDied","Data":"c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28"}
Oct 30 01:24:38.780556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.780413    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6" event={"ID":"ee8064a2-6df2-402b-a87b-f10caa63c39a","Type":"ContainerDied","Data":"1c50954d601f75c3fad0e9855c49fad6d522ad766575eaccaefd7d42a41578e3"}
Oct 30 01:24:38.780556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.780440    6011 scope.go:122] "RemoveContainer" containerID="c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28"
Oct 30 01:24:38.791903 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.791883    6011 scope.go:122] "RemoveContainer" containerID="c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28"
Oct 30 01:24:38.792262 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:38.792244    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28\": container with ID starting with c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28 not found: ID does not exist" containerID="c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28"
Oct 30 01:24:38.792336 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.792270    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28"} err="failed to get container status \"c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28\": rpc error: code = NotFound desc = could not find container \"c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28\": container with ID starting with c86b42e30a586f26f7af6e3b39b0971a4dc6af1d0578437e64a6d529a2384f28 not found: ID does not exist"
Oct 30 01:24:38.837831 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.837787    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"]
Oct 30 01:24:38.850798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:38.850758    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-4b973b-5885c4f696-flqw6"]
Oct 30 01:24:40.021615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:40.021571    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee8064a2-6df2-402b-a87b-f10caa63c39a" path="/var/lib/kubelet/pods/ee8064a2-6df2-402b-a87b-f10caa63c39a/volumes"
Oct 30 01:24:41.754141 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:41.754112    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:24:43.366660 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.366620    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"]
Oct 30 01:24:43.367619 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.366907    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x" podUID="b34b3c21-b0d9-4492-9c42-81ed216616d6" containerName="maas-api" containerID="cri-o://c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b" gracePeriod=30
Oct 30 01:24:43.667752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.667721    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:24:43.745059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.744973    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-metrics-certs\") pod \"b34b3c21-b0d9-4492-9c42-81ed216616d6\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") "
Oct 30 01:24:43.745294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.745108    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b34b3c21-b0d9-4492-9c42-81ed216616d6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34b3c21-b0d9-4492-9c42-81ed216616d6-openshift-service-ca\") pod \"b34b3c21-b0d9-4492-9c42-81ed216616d6\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") "
Oct 30 01:24:43.745367 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.745348    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-maas-api-tls\") pod \"b34b3c21-b0d9-4492-9c42-81ed216616d6\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") "
Oct 30 01:24:43.745428 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.745404    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b34b3c21-b0d9-4492-9c42-81ed216616d6-kube-api-access-r68k6\" (UniqueName: \"kubernetes.io/projected/b34b3c21-b0d9-4492-9c42-81ed216616d6-kube-api-access-r68k6\") pod \"b34b3c21-b0d9-4492-9c42-81ed216616d6\" (UID: \"b34b3c21-b0d9-4492-9c42-81ed216616d6\") "
Oct 30 01:24:43.746089 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.746053    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b34b3c21-b0d9-4492-9c42-81ed216616d6-openshift-service-ca" pod "b34b3c21-b0d9-4492-9c42-81ed216616d6" (UID: "b34b3c21-b0d9-4492-9c42-81ed216616d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:24:43.746538 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.746512    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b34b3c21-b0d9-4492-9c42-81ed216616d6-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:43.748811 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.748775    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b34b3c21-b0d9-4492-9c42-81ed216616d6-kube-api-access-r68k6" pod "b34b3c21-b0d9-4492-9c42-81ed216616d6" (UID: "b34b3c21-b0d9-4492-9c42-81ed216616d6"). InnerVolumeSpecName "kube-api-access-r68k6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:24:43.755862 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.755822    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-metrics-certs" pod "b34b3c21-b0d9-4492-9c42-81ed216616d6" (UID: "b34b3c21-b0d9-4492-9c42-81ed216616d6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:43.757800 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.757761    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-maas-api-tls" pod "b34b3c21-b0d9-4492-9c42-81ed216616d6" (UID: "b34b3c21-b0d9-4492-9c42-81ed216616d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:24:43.838339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.838071    6011 generic.go:356] "Generic (PLEG): container finished" podID="b34b3c21-b0d9-4492-9c42-81ed216616d6" containerID="c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b" exitCode=0
Oct 30 01:24:43.838550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.838348    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x" event={"ID":"b34b3c21-b0d9-4492-9c42-81ed216616d6","Type":"ContainerDied","Data":"c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b"}
Oct 30 01:24:43.838550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.838408    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x" event={"ID":"b34b3c21-b0d9-4492-9c42-81ed216616d6","Type":"ContainerDied","Data":"0773ac6c247fa36421bc613c03a65df3eac72e81baeb0b0ac4d88126227b5625"}
Oct 30 01:24:43.838550 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.838439    6011 scope.go:122] "RemoveContainer" containerID="c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b"
Oct 30 01:24:43.838767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.838602    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"
Oct 30 01:24:43.853640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.853281    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:43.853640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.853314    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r68k6\" (UniqueName: \"kubernetes.io/projected/b34b3c21-b0d9-4492-9c42-81ed216616d6-kube-api-access-r68k6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:43.853640 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.853331    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b34b3c21-b0d9-4492-9c42-81ed216616d6-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:24:43.870646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.870612    6011 scope.go:122] "RemoveContainer" containerID="c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b"
Oct 30 01:24:43.872471 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:43.872443    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b\": container with ID starting with c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b not found: ID does not exist" containerID="c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b"
Oct 30 01:24:43.872659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.872635    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b"} err="failed to get container status \"c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b\": rpc error: code = NotFound desc = could not find container \"c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b\": container with ID starting with c5ba6ffb109060db4d5ca9f84db788c15b85aaf6c8e4ff35164427a95741d75b not found: ID does not exist"
Oct 30 01:24:43.896855 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.896736    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"]
Oct 30 01:24:43.900376 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:43.900342    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-cfe8f0-dd858dbfb-zhl5x"]
Oct 30 01:24:44.021883 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:24:44.021848    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b34b3c21-b0d9-4492-9c42-81ed216616d6" path="/var/lib/kubelet/pods/b34b3c21-b0d9-4492-9c42-81ed216616d6/volumes"
Oct 30 01:24:57.911288 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:24:57.911246    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:25:02.511976 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:02.511938    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"]
Oct 30 01:25:02.512862 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:02.512825    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy" containerID="cri-o://1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f" gracePeriod=30
Oct 30 01:25:02.618935 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:02.618895    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"]
Oct 30 01:25:02.681199 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:02.680420    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"]
Oct 30 01:25:07.620285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.620231    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" podUID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" containerName="payload-processing" containerID="cri-o://6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2" gracePeriod=25
Oct 30 01:25:07.680978 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.680906    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" podUID="df652682-5a54-4e2e-afdb-25d2dffe9953" containerName="payload-pre-processing" containerID="cri-o://766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f" gracePeriod=25
Oct 30 01:25:07.809846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.809775    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:25:07.959477 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959439    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a922739c-1386-4b81-86f9-4cd0bc393387-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a922739c-1386-4b81-86f9-4cd0bc393387-istiod-ca-cert\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959491    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-credential-socket\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959532    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-certs\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959584    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-envoy\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959634    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a922739c-1386-4b81-86f9-4cd0bc393387-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a922739c-1386-4b81-86f9-4cd0bc393387-istio-podinfo\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959703    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-istio-token\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-istio-token\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959736    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-socket\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959766    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-data\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959851    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-kube-api-access-98krq\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-kube-api-access-98krq\") pod \"a922739c-1386-4b81-86f9-4cd0bc393387\" (UID: \"a922739c-1386-4b81-86f9-4cd0bc393387\") "
Oct 30 01:25:07.959902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959872    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-credential-socket" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:07.960163 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.959918    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a922739c-1386-4b81-86f9-4cd0bc393387-istiod-ca-cert" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:07.960243 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.960223    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-data" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:07.960305 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.960243    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-socket" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:07.960538 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.960514    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-certs" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:07.960950 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.960929    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a922739c-1386-4b81-86f9-4cd0bc393387-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:07.961049 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.960956    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:07.961049 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.960970    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:07.961049 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.960983    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:07.961049 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.961000    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:07.962499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.962365    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-envoy" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:07.962499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.962470    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-istio-token" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:07.962499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.962489    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-kube-api-access-98krq" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "kube-api-access-98krq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:07.962675 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:07.962508    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a922739c-1386-4b81-86f9-4cd0bc393387-istio-podinfo" pod "a922739c-1386-4b81-86f9-4cd0bc393387" (UID: "a922739c-1386-4b81-86f9-4cd0bc393387"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:25:08.057864 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:08.057826    6011 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda922739c_1386_4b81_86f9_4cd0bc393387.slice/crio-a1999b30361888d296cff30fd7b77dbc97821b11cffe8f69330d4ffecc77cbd0\": RecentStats: unable to find data in memory cache]"
Oct 30 01:25:08.061941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.061850    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98krq\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-kube-api-access-98krq\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:08.061941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.061885    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a922739c-1386-4b81-86f9-4cd0bc393387-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:08.061941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.061896    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a922739c-1386-4b81-86f9-4cd0bc393387-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:08.061941 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.061905    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a922739c-1386-4b81-86f9-4cd0bc393387-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:08.069461 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.069393    6011 generic.go:356] "Generic (PLEG): container finished" podID="a922739c-1386-4b81-86f9-4cd0bc393387" containerID="1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f" exitCode=0
Oct 30 01:25:08.069667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.069648    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" event={"ID":"a922739c-1386-4b81-86f9-4cd0bc393387","Type":"ContainerDied","Data":"1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f"}
Oct 30 01:25:08.069765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.069725    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv" event={"ID":"a922739c-1386-4b81-86f9-4cd0bc393387","Type":"ContainerDied","Data":"a1999b30361888d296cff30fd7b77dbc97821b11cffe8f69330d4ffecc77cbd0"}
Oct 30 01:25:08.069765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.069748    6011 scope.go:122] "RemoveContainer" containerID="1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f"
Oct 30 01:25:08.069913 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.069900    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"
Oct 30 01:25:08.088531 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.088503    6011 scope.go:122] "RemoveContainer" containerID="1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f"
Oct 30 01:25:08.088929 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:08.088897    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f\": container with ID starting with 1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f not found: ID does not exist" containerID="1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f"
Oct 30 01:25:08.089003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.088944    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f"} err="failed to get container status \"1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f\": rpc error: code = NotFound desc = could not find container \"1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f\": container with ID starting with 1c4f6513411a104a0e5f95ea7f42519df59e3005755007ea19e58cab2385054f not found: ID does not exist"
Oct 30 01:25:08.128857 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.128818    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"]
Oct 30 01:25:08.131707 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:08.131677    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-d0ae0d-openshift-default-6f7cd449cc-bwgjv"]
Oct 30 01:25:10.021741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:10.021694    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" path="/var/lib/kubelet/pods/a922739c-1386-4b81-86f9-4cd0bc393387/volumes"
Oct 30 01:25:10.711346 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:10.711266    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" podUID="df652682-5a54-4e2e-afdb-25d2dffe9953" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:10.711758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:10.711446    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" podUID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:14.625652 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:14.625616    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"]
Oct 30 01:25:14.660305 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:14.660265    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"]
Oct 30 01:25:19.626849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:19.626792    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" podUID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" containerName="payload-processing" containerID="cri-o://01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01" gracePeriod=25
Oct 30 01:25:19.661375 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:19.661321    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" podUID="db66c456-50da-4fa8-9361-24090e0fa74f" containerName="payload-pre-processing" containerID="cri-o://bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda" gracePeriod=25
Oct 30 01:25:20.711697 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:20.711648    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" podUID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:20.712134 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:20.711741    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" podUID="df652682-5a54-4e2e-afdb-25d2dffe9953" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:21.718816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:21.718773    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" podUID="db66c456-50da-4fa8-9361-24090e0fa74f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:21.729611 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:21.729563    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"]
Oct 30 01:25:21.729808 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:21.729777    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6" podUID="64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" containerName="maas-api" containerID="cri-o://467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c" gracePeriod=30
Oct 30 01:25:22.054012 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.053965    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 01:25:22.135338 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.135287    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls\") pod \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") "
Oct 30 01:25:22.135564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.135378    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-kube-api-access-rfqbj\" (UniqueName: \"kubernetes.io/projected/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-kube-api-access-rfqbj\") pod \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") "
Oct 30 01:25:22.135564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.135526    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-metrics-certs\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-metrics-certs\") pod \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") "
Oct 30 01:25:22.135702 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.135597    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-openshift-service-ca\") pod \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\" (UID: \"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf\") "
Oct 30 01:25:22.136022 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.135984    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-openshift-service-ca" pod "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" (UID: "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:22.136537 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.136511    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:22.137988 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.137953    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls" pod "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" (UID: "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:22.138430 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.138401    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-metrics-certs" pod "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" (UID: "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:22.138571 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.138545    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-kube-api-access-rfqbj" pod "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" (UID: "64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf"). InnerVolumeSpecName "kube-api-access-rfqbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:22.171762 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.171719    6011 generic.go:356] "Generic (PLEG): container finished" podID="64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" containerID="467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c" exitCode=0
Oct 30 01:25:22.171948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.171772    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"
Oct 30 01:25:22.171948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.171794    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6" event={"ID":"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf","Type":"ContainerDied","Data":"467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c"}
Oct 30 01:25:22.171948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.171855    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6" event={"ID":"64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf","Type":"ContainerDied","Data":"311bfbd3f5e068ff2f0fd9aafc2f1d836b0e5608632db8492edcb14440953e12"}
Oct 30 01:25:22.171948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.171886    6011 scope.go:122] "RemoveContainer" containerID="467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c"
Oct 30 01:25:22.185273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.185247    6011 scope.go:122] "RemoveContainer" containerID="467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c"
Oct 30 01:25:22.185689 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:22.185666    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c\": container with ID starting with 467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c not found: ID does not exist" containerID="467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c"
Oct 30 01:25:22.185792 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.185697    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c"} err="failed to get container status \"467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c\": rpc error: code = NotFound desc = could not find container \"467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c\": container with ID starting with 467474a4ce925f09ce24371a13db8d56ac7828eb4d573699c59bd287c397ff3c not found: ID does not exist"
Oct 30 01:25:22.238201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.229136    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"]
Oct 30 01:25:22.249233 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.238539    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:22.249233 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.238585    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfqbj\" (UniqueName: \"kubernetes.io/projected/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-kube-api-access-rfqbj\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:22.249233 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.238601    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:22.273554 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.269590    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-298ddc-78dd9677d7-4k7p6"]
Oct 30 01:25:22.725108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:22.725063    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" podUID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:24.022637 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:24.022586    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" path="/var/lib/kubelet/pods/64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf/volumes"
Oct 30 01:25:27.913520 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:27.913435    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:25:27.969101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:27.969064    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 01:25:28.035734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.034474    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-plugins-config-volume\") pod \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") "
Oct 30 01:25:28.035734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.034698    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-kube-api-access-ns752\" (UniqueName: \"kubernetes.io/projected/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-kube-api-access-ns752\") pod \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\" (UID: \"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0\") "
Oct 30 01:25:28.035734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.035685    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-plugins-config-volume" pod "a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" (UID: "a05b1685-5535-4c9c-a9cd-cb55f0c79cb0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:28.038998 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.038927    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-kube-api-access-ns752" pod "a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" (UID: "a05b1685-5535-4c9c-a9cd-cb55f0c79cb0"). InnerVolumeSpecName "kube-api-access-ns752". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:28.043869 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.043843    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 01:25:28.136368 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.136321    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df652682-5a54-4e2e-afdb-25d2dffe9953-kube-api-access-xrjzm\" (UniqueName: \"kubernetes.io/projected/df652682-5a54-4e2e-afdb-25d2dffe9953-kube-api-access-xrjzm\") pod \"df652682-5a54-4e2e-afdb-25d2dffe9953\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") "
Oct 30 01:25:28.136578 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.136532    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df652682-5a54-4e2e-afdb-25d2dffe9953-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df652682-5a54-4e2e-afdb-25d2dffe9953-plugins-config-volume\") pod \"df652682-5a54-4e2e-afdb-25d2dffe9953\" (UID: \"df652682-5a54-4e2e-afdb-25d2dffe9953\") "
Oct 30 01:25:28.136902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.136876    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df652682-5a54-4e2e-afdb-25d2dffe9953-plugins-config-volume" pod "df652682-5a54-4e2e-afdb-25d2dffe9953" (UID: "df652682-5a54-4e2e-afdb-25d2dffe9953"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:28.137015 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.137001    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns752\" (UniqueName: \"kubernetes.io/projected/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-kube-api-access-ns752\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:28.137067 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.137018    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:28.137067 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.137028    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/df652682-5a54-4e2e-afdb-25d2dffe9953-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:28.138595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.138573    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df652682-5a54-4e2e-afdb-25d2dffe9953-kube-api-access-xrjzm" pod "df652682-5a54-4e2e-afdb-25d2dffe9953" (UID: "df652682-5a54-4e2e-afdb-25d2dffe9953"). InnerVolumeSpecName "kube-api-access-xrjzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:28.252652 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.244412    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrjzm\" (UniqueName: \"kubernetes.io/projected/df652682-5a54-4e2e-afdb-25d2dffe9953-kube-api-access-xrjzm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:28.288874 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.288404    6011 generic.go:356] "Generic (PLEG): container finished" podID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" containerID="6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2" exitCode=0
Oct 30 01:25:28.288874 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.288534    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" event={"ID":"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0","Type":"ContainerDied","Data":"6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2"}
Oct 30 01:25:28.288874 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.288628    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5" event={"ID":"a05b1685-5535-4c9c-a9cd-cb55f0c79cb0","Type":"ContainerDied","Data":"d65d7109fe0561a460663338f82d0584402ae983d434af833cf6fc1b0e75b3bd"}
Oct 30 01:25:28.288874 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.288663    6011 scope.go:122] "RemoveContainer" containerID="6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2"
Oct 30 01:25:28.288874 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.288851    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"
Oct 30 01:25:28.305475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.305432    6011 generic.go:356] "Generic (PLEG): container finished" podID="df652682-5a54-4e2e-afdb-25d2dffe9953" containerID="766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f" exitCode=0
Oct 30 01:25:28.305646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.305633    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" event={"ID":"df652682-5a54-4e2e-afdb-25d2dffe9953","Type":"ContainerDied","Data":"766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f"}
Oct 30 01:25:28.305718 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.305675    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj" event={"ID":"df652682-5a54-4e2e-afdb-25d2dffe9953","Type":"ContainerDied","Data":"7d6a12a4cc5fd3376cb19898159af3df563ce59ed56cdf9188178006c057395e"}
Oct 30 01:25:28.305840 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.305812    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"
Oct 30 01:25:28.317435 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.317304    6011 scope.go:122] "RemoveContainer" containerID="6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2"
Oct 30 01:25:28.318152 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:28.317981    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2\": container with ID starting with 6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2 not found: ID does not exist" containerID="6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2"
Oct 30 01:25:28.318152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.318025    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2"} err="failed to get container status \"6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2\": rpc error: code = NotFound desc = could not find container \"6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2\": container with ID starting with 6f19417383bd50f9562285530a2e507ead8a5cee8a6fc77ba61f919e92c55fc2 not found: ID does not exist"
Oct 30 01:25:28.318152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.318054    6011 scope.go:122] "RemoveContainer" containerID="766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f"
Oct 30 01:25:28.340502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.340465    6011 scope.go:122] "RemoveContainer" containerID="766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f"
Oct 30 01:25:28.341037 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:28.340948    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f\": container with ID starting with 766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f not found: ID does not exist" containerID="766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f"
Oct 30 01:25:28.341037 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.340994    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f"} err="failed to get container status \"766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f\": rpc error: code = NotFound desc = could not find container \"766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f\": container with ID starting with 766f08e65241301772b84805ad162a3f6c52f1fa4acee6f35c06cc915807019f not found: ID does not exist"
Oct 30 01:25:28.350488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.349549    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"]
Oct 30 01:25:28.359850 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.359809    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w2-7d2b75-6d9689fc96-mt8c5"]
Oct 30 01:25:28.402302 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.402250    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"]
Oct 30 01:25:28.405162 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:28.405129    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-7d2b75-5f8d79845f-k9ncj"]
Oct 30 01:25:29.889657 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:29.889606    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"]
Oct 30 01:25:29.890228 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:29.889892    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy" containerID="cri-o://cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e" gracePeriod=30
Oct 30 01:25:30.053301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:30.052143    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" path="/var/lib/kubelet/pods/a05b1685-5535-4c9c-a9cd-cb55f0c79cb0/volumes"
Oct 30 01:25:30.053301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:30.052881    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df652682-5a54-4e2e-afdb-25d2dffe9953" path="/var/lib/kubelet/pods/df652682-5a54-4e2e-afdb-25d2dffe9953/volumes"
Oct 30 01:25:31.718733 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:31.718695    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" podUID="db66c456-50da-4fa8-9361-24090e0fa74f" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:32.727131 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:32.727080    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" podUID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:25:33.745763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:33.745715    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"]
Oct 30 01:25:33.746325 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:33.745963    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w" podUID="c094f8f4-a12e-489e-92b3-908016822212" containerName="maas-api" containerID="cri-o://332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61" gracePeriod=30
Oct 30 01:25:34.037573 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.037527    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:25:34.160475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.160432    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c094f8f4-a12e-489e-92b3-908016822212-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094f8f4-a12e-489e-92b3-908016822212-openshift-service-ca\") pod \"c094f8f4-a12e-489e-92b3-908016822212\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") "
Oct 30 01:25:34.160682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.160489    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c094f8f4-a12e-489e-92b3-908016822212-kube-api-access-r9dpt\" (UniqueName: \"kubernetes.io/projected/c094f8f4-a12e-489e-92b3-908016822212-kube-api-access-r9dpt\") pod \"c094f8f4-a12e-489e-92b3-908016822212\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") "
Oct 30 01:25:34.160682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.160558    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-metrics-certs\") pod \"c094f8f4-a12e-489e-92b3-908016822212\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") "
Oct 30 01:25:34.160682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.160637    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-maas-api-tls\") pod \"c094f8f4-a12e-489e-92b3-908016822212\" (UID: \"c094f8f4-a12e-489e-92b3-908016822212\") "
Oct 30 01:25:34.160915 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.160888    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c094f8f4-a12e-489e-92b3-908016822212-openshift-service-ca" pod "c094f8f4-a12e-489e-92b3-908016822212" (UID: "c094f8f4-a12e-489e-92b3-908016822212"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:34.161643 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.161594    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c094f8f4-a12e-489e-92b3-908016822212-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:34.163090 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.163065    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-metrics-certs" pod "c094f8f4-a12e-489e-92b3-908016822212" (UID: "c094f8f4-a12e-489e-92b3-908016822212"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:34.163454 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.163428    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-maas-api-tls" pod "c094f8f4-a12e-489e-92b3-908016822212" (UID: "c094f8f4-a12e-489e-92b3-908016822212"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:34.163544 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.163456    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c094f8f4-a12e-489e-92b3-908016822212-kube-api-access-r9dpt" pod "c094f8f4-a12e-489e-92b3-908016822212" (UID: "c094f8f4-a12e-489e-92b3-908016822212"). InnerVolumeSpecName "kube-api-access-r9dpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:34.262928 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.262843    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:34.262928 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.262879    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c094f8f4-a12e-489e-92b3-908016822212-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:34.262928 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.262890    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9dpt\" (UniqueName: \"kubernetes.io/projected/c094f8f4-a12e-489e-92b3-908016822212-kube-api-access-r9dpt\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:34.350039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.350003    6011 generic.go:356] "Generic (PLEG): container finished" podID="c094f8f4-a12e-489e-92b3-908016822212" containerID="332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61" exitCode=0
Oct 30 01:25:34.350283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.350062    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w" event={"ID":"c094f8f4-a12e-489e-92b3-908016822212","Type":"ContainerDied","Data":"332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61"}
Oct 30 01:25:34.350283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.350077    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"
Oct 30 01:25:34.350283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.350115    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w" event={"ID":"c094f8f4-a12e-489e-92b3-908016822212","Type":"ContainerDied","Data":"9fad240977e6d01331fe1a84b2467555b9dd81c9fb883a953d941f3d9cd874ec"}
Oct 30 01:25:34.350283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.350145    6011 scope.go:122] "RemoveContainer" containerID="332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61"
Oct 30 01:25:34.362782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.362760    6011 scope.go:122] "RemoveContainer" containerID="332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61"
Oct 30 01:25:34.363362 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:34.363327    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61\": container with ID starting with 332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61 not found: ID does not exist" containerID="332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61"
Oct 30 01:25:34.363727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.363371    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61"} err="failed to get container status \"332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61\": rpc error: code = NotFound desc = could not find container \"332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61\": container with ID starting with 332149281ece57457fe4fafa6a3d9a93f8251847af0746b7b21f8141a91b4d61 not found: ID does not exist"
Oct 30 01:25:34.406799 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.406769    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"]
Oct 30 01:25:34.411816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:34.411777    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-11fbe8-744fbb5f8-rmp5w"]
Oct 30 01:25:35.178500 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.178436    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 01:25:35.281985 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.281951    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-socket\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282021    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-token\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-token\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282068    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-credential-socket\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282129    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-kube-api-access-rqqnm\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-kube-api-access-rqqnm\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282160 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282157    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-podinfo\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282457 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282219    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-data\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282457 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282262    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-certs\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282457 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282291    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istiod-ca-cert\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282457 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282340    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-socket" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:35.282457 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282361    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-envoy\") pod \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\" (UID: \"9a877407-b50a-41e5-91af-7d9cf4d2fe53\") "
Oct 30 01:25:35.282756 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282730    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-credential-socket" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:35.282816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282747    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-data" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:35.282816 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.282801    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.285071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.285046    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-token" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:35.285238 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.285069    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-podinfo" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:25:35.285464 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.285361    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-envoy" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:35.288110 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.288051    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-kube-api-access-rqqnm" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "kube-api-access-rqqnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:35.288424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.288399    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-certs" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:25:35.288514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.288461    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istiod-ca-cert" pod "9a877407-b50a-41e5-91af-7d9cf4d2fe53" (UID: "9a877407-b50a-41e5-91af-7d9cf4d2fe53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:35.358734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.358695    6011 generic.go:356] "Generic (PLEG): container finished" podID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerID="cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e" exitCode=0
Oct 30 01:25:35.358946 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.358749    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"
Oct 30 01:25:35.358946 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.358777    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" event={"ID":"9a877407-b50a-41e5-91af-7d9cf4d2fe53","Type":"ContainerDied","Data":"cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e"}
Oct 30 01:25:35.358946 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.358834    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz" event={"ID":"9a877407-b50a-41e5-91af-7d9cf4d2fe53","Type":"ContainerDied","Data":"b2858af64e493382a17c058df183f981a637358f61d416bce536c85456518279"}
Oct 30 01:25:35.358946 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.358861    6011 scope.go:122] "RemoveContainer" containerID="cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e"
Oct 30 01:25:35.370316 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.370296    6011 scope.go:122] "RemoveContainer" containerID="cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e"
Oct 30 01:25:35.370635 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:35.370619    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e\": container with ID starting with cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e not found: ID does not exist" containerID="cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e"
Oct 30 01:25:35.370683 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.370643    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e"} err="failed to get container status \"cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e\": rpc error: code = NotFound desc = could not find container \"cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e\": container with ID starting with cab6a4fe0874e8a407448b82055096517df5fc8df5eee95bc15ee892a7f0515e not found: ID does not exist"
Oct 30 01:25:35.384247 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384216    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqqnm\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-kube-api-access-rqqnm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.384247 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384249    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.384432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384263    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.384432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384275    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.384432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384286    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.384432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384297    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.384432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384307    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a877407-b50a-41e5-91af-7d9cf4d2fe53-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.384432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.384321    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a877407-b50a-41e5-91af-7d9cf4d2fe53-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:35.417274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.417243    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"]
Oct 30 01:25:35.422851 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:35.422824    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-298ddc-openshift-default-64b9897bb7-bb2zz"]
Oct 30 01:25:36.022075 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:36.022042    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" path="/var/lib/kubelet/pods/9a877407-b50a-41e5-91af-7d9cf4d2fe53/volumes"
Oct 30 01:25:36.022572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:36.022554    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c094f8f4-a12e-489e-92b3-908016822212" path="/var/lib/kubelet/pods/c094f8f4-a12e-489e-92b3-908016822212/volumes"
Oct 30 01:25:39.945462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:39.945428    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 01:25:39.977377 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:39.977354    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 01:25:40.046011 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.045963    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/db66c456-50da-4fa8-9361-24090e0fa74f-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66c456-50da-4fa8-9361-24090e0fa74f-plugins-config-volume\") pod \"db66c456-50da-4fa8-9361-24090e0fa74f\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") "
Oct 30 01:25:40.046236 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.046045    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a414233b-c7f4-45bf-bb2a-837e6ae8203a-kube-api-access-62bfv\" (UniqueName: \"kubernetes.io/projected/a414233b-c7f4-45bf-bb2a-837e6ae8203a-kube-api-access-62bfv\") pod \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") "
Oct 30 01:25:40.046236 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.046212    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a414233b-c7f4-45bf-bb2a-837e6ae8203a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a414233b-c7f4-45bf-bb2a-837e6ae8203a-plugins-config-volume\") pod \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\" (UID: \"a414233b-c7f4-45bf-bb2a-837e6ae8203a\") "
Oct 30 01:25:40.046382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.046293    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/db66c456-50da-4fa8-9361-24090e0fa74f-kube-api-access-2g6tt\" (UniqueName: \"kubernetes.io/projected/db66c456-50da-4fa8-9361-24090e0fa74f-kube-api-access-2g6tt\") pod \"db66c456-50da-4fa8-9361-24090e0fa74f\" (UID: \"db66c456-50da-4fa8-9361-24090e0fa74f\") "
Oct 30 01:25:40.046443 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.046420    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db66c456-50da-4fa8-9361-24090e0fa74f-plugins-config-volume" pod "db66c456-50da-4fa8-9361-24090e0fa74f" (UID: "db66c456-50da-4fa8-9361-24090e0fa74f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:40.046823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.046796    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/db66c456-50da-4fa8-9361-24090e0fa74f-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:40.047086 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.046939    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a414233b-c7f4-45bf-bb2a-837e6ae8203a-plugins-config-volume" pod "a414233b-c7f4-45bf-bb2a-837e6ae8203a" (UID: "a414233b-c7f4-45bf-bb2a-837e6ae8203a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:40.048573 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.048546    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a414233b-c7f4-45bf-bb2a-837e6ae8203a-kube-api-access-62bfv" pod "a414233b-c7f4-45bf-bb2a-837e6ae8203a" (UID: "a414233b-c7f4-45bf-bb2a-837e6ae8203a"). InnerVolumeSpecName "kube-api-access-62bfv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:40.048753 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.048733    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db66c456-50da-4fa8-9361-24090e0fa74f-kube-api-access-2g6tt" pod "db66c456-50da-4fa8-9361-24090e0fa74f" (UID: "db66c456-50da-4fa8-9361-24090e0fa74f"). InnerVolumeSpecName "kube-api-access-2g6tt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:40.147880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.147840    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62bfv\" (UniqueName: \"kubernetes.io/projected/a414233b-c7f4-45bf-bb2a-837e6ae8203a-kube-api-access-62bfv\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:40.147880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.147873    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a414233b-c7f4-45bf-bb2a-837e6ae8203a-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:40.147880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.147885    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2g6tt\" (UniqueName: \"kubernetes.io/projected/db66c456-50da-4fa8-9361-24090e0fa74f-kube-api-access-2g6tt\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:40.409025 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.408984    6011 generic.go:356] "Generic (PLEG): container finished" podID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" containerID="01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01" exitCode=0
Oct 30 01:25:40.409208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.409038    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"
Oct 30 01:25:40.409208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.409075    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" event={"ID":"a414233b-c7f4-45bf-bb2a-837e6ae8203a","Type":"ContainerDied","Data":"01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01"}
Oct 30 01:25:40.409208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.409133    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz" event={"ID":"a414233b-c7f4-45bf-bb2a-837e6ae8203a","Type":"ContainerDied","Data":"89c09469b391b27ab8f7a7e5c10134c987ba54f9f5b937da09e48fa31ac34997"}
Oct 30 01:25:40.409208 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.409163    6011 scope.go:122] "RemoveContainer" containerID="01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01"
Oct 30 01:25:40.410658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.410638    6011 generic.go:356] "Generic (PLEG): container finished" podID="db66c456-50da-4fa8-9361-24090e0fa74f" containerID="bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda" exitCode=0
Oct 30 01:25:40.410770 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.410752    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" event={"ID":"db66c456-50da-4fa8-9361-24090e0fa74f","Type":"ContainerDied","Data":"bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda"}
Oct 30 01:25:40.410817 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.410796    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc" event={"ID":"db66c456-50da-4fa8-9361-24090e0fa74f","Type":"ContainerDied","Data":"7a7876972ebcb92a94cf30ef8bac3a8db00a07061e7e0de31fddb7dd8aaf61b2"}
Oct 30 01:25:40.410925 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.410904    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"
Oct 30 01:25:40.421049 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.421033    6011 scope.go:122] "RemoveContainer" containerID="01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01"
Oct 30 01:25:40.421393 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:40.421367    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01\": container with ID starting with 01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01 not found: ID does not exist" containerID="01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01"
Oct 30 01:25:40.421505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.421395    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01"} err="failed to get container status \"01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01\": rpc error: code = NotFound desc = could not find container \"01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01\": container with ID starting with 01c8c3dc12d6c7ba9a7825170cd0e9c10dd28802f59a544a4fe171f0eb7f9f01 not found: ID does not exist"
Oct 30 01:25:40.421505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.421411    6011 scope.go:122] "RemoveContainer" containerID="bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda"
Oct 30 01:25:40.430993 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.430971    6011 scope.go:122] "RemoveContainer" containerID="bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda"
Oct 30 01:25:40.431366 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:40.431344    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda\": container with ID starting with bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda not found: ID does not exist" containerID="bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda"
Oct 30 01:25:40.431418 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.431375    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda"} err="failed to get container status \"bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda\": rpc error: code = NotFound desc = could not find container \"bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda\": container with ID starting with bc59a9aa65c21c416aae7a540beb95b0693af4bf97faaf3a6af27a0513ef1bda not found: ID does not exist"
Oct 30 01:25:40.463336 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.463300    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"]
Oct 30 01:25:40.476580 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.476535    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w5-da76a2-77fcbffd7-sjwkc"]
Oct 30 01:25:40.503756 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.503719    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"]
Oct 30 01:25:40.508275 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:40.508241    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w5-da76a2-87d968d7d-wb5xz"]
Oct 30 01:25:42.021934 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:42.021895    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" path="/var/lib/kubelet/pods/a414233b-c7f4-45bf-bb2a-837e6ae8203a/volumes"
Oct 30 01:25:42.022475 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:42.022352    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="db66c456-50da-4fa8-9361-24090e0fa74f" path="/var/lib/kubelet/pods/db66c456-50da-4fa8-9361-24090e0fa74f/volumes"
Oct 30 01:25:49.349144 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.349106    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"]
Oct 30 01:25:49.349808 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.349422    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd" podUID="5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" containerName="maas-api" containerID="cri-o://423f8ea4dc7c89e7d421fba40660cd80c256f7264855b3c6d3f3751a87b9ad53" gracePeriod=30
Oct 30 01:25:49.477279 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.477241    6011 generic.go:356] "Generic (PLEG): container finished" podID="5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" containerID="423f8ea4dc7c89e7d421fba40660cd80c256f7264855b3c6d3f3751a87b9ad53" exitCode=0
Oct 30 01:25:49.477405 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.477305    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd" event={"ID":"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d","Type":"ContainerDied","Data":"423f8ea4dc7c89e7d421fba40660cd80c256f7264855b3c6d3f3751a87b9ad53"}
Oct 30 01:25:49.661262 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.661237    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 01:25:49.805625 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.805587    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-kube-api-access-tshdb\" (UniqueName: \"kubernetes.io/projected/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-kube-api-access-tshdb\") pod \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") "
Oct 30 01:25:49.805841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.805716    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-metrics-certs\") pod \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") "
Oct 30 01:25:49.805841 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.805808    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-openshift-service-ca\") pod \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") "
Oct 30 01:25:49.805983 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.805865    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-maas-api-tls\") pod \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\" (UID: \"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d\") "
Oct 30 01:25:49.806242 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.806208    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-openshift-service-ca" pod "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" (UID: "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:49.806551 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.806524    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:49.808091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.808066    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-kube-api-access-tshdb" pod "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" (UID: "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d"). InnerVolumeSpecName "kube-api-access-tshdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:49.808237 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.808213    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-maas-api-tls" pod "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" (UID: "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:49.808335 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.808295    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-metrics-certs" pod "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" (UID: "5500c7ef-d6bc-4a85-b30d-b57e3e7af43d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:49.907596 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.907514    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tshdb\" (UniqueName: \"kubernetes.io/projected/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-kube-api-access-tshdb\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:49.907596 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.907541    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:49.907596 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:49.907552    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:50.485846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:50.485817    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"
Oct 30 01:25:50.485846 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:50.485833    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd" event={"ID":"5500c7ef-d6bc-4a85-b30d-b57e3e7af43d","Type":"ContainerDied","Data":"2e19df1e804ca6d6f0b401303487c416af958a8f0a72ba0d94a778465ec42f32"}
Oct 30 01:25:50.486364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:50.485893    6011 scope.go:122] "RemoveContainer" containerID="423f8ea4dc7c89e7d421fba40660cd80c256f7264855b3c6d3f3751a87b9ad53"
Oct 30 01:25:50.540291 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:50.540251    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"]
Oct 30 01:25:50.553394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:50.553352    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-7d2b75-d97bc9cf7-k8zhd"]
Oct 30 01:25:52.022432 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.022401    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" path="/var/lib/kubelet/pods/5500c7ef-d6bc-4a85-b30d-b57e3e7af43d/volumes"
Oct 30 01:25:52.032779 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.032745    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"]
Oct 30 01:25:52.033003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.032970    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln" podUID="bd1152b0-ccfa-4507-b220-231ee84d3790" containerName="maas-api" containerID="cri-o://13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3" gracePeriod=30
Oct 30 01:25:52.327953 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.327921    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 01:25:52.443711 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.443669    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bd1152b0-ccfa-4507-b220-231ee84d3790-kube-api-access-frzf4\" (UniqueName: \"kubernetes.io/projected/bd1152b0-ccfa-4507-b220-231ee84d3790-kube-api-access-frzf4\") pod \"bd1152b0-ccfa-4507-b220-231ee84d3790\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") "
Oct 30 01:25:52.449058 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.443981    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-maas-api-tls\") pod \"bd1152b0-ccfa-4507-b220-231ee84d3790\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") "
Oct 30 01:25:52.449544 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.449514    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bd1152b0-ccfa-4507-b220-231ee84d3790-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd1152b0-ccfa-4507-b220-231ee84d3790-openshift-service-ca\") pod \"bd1152b0-ccfa-4507-b220-231ee84d3790\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") "
Oct 30 01:25:52.449735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.449719    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-metrics-certs\") pod \"bd1152b0-ccfa-4507-b220-231ee84d3790\" (UID: \"bd1152b0-ccfa-4507-b220-231ee84d3790\") "
Oct 30 01:25:52.454202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.450385    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd1152b0-ccfa-4507-b220-231ee84d3790-openshift-service-ca" pod "bd1152b0-ccfa-4507-b220-231ee84d3790" (UID: "bd1152b0-ccfa-4507-b220-231ee84d3790"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:52.454623 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.454601    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd1152b0-ccfa-4507-b220-231ee84d3790-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:52.457327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.457285    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-maas-api-tls" pod "bd1152b0-ccfa-4507-b220-231ee84d3790" (UID: "bd1152b0-ccfa-4507-b220-231ee84d3790"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:52.467761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.467721    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-metrics-certs" pod "bd1152b0-ccfa-4507-b220-231ee84d3790" (UID: "bd1152b0-ccfa-4507-b220-231ee84d3790"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:52.467906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.467849    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd1152b0-ccfa-4507-b220-231ee84d3790-kube-api-access-frzf4" pod "bd1152b0-ccfa-4507-b220-231ee84d3790" (UID: "bd1152b0-ccfa-4507-b220-231ee84d3790"). InnerVolumeSpecName "kube-api-access-frzf4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:52.503103 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.503068    6011 generic.go:356] "Generic (PLEG): container finished" podID="bd1152b0-ccfa-4507-b220-231ee84d3790" containerID="13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3" exitCode=0
Oct 30 01:25:52.503344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.503134    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"
Oct 30 01:25:52.503344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.503152    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln" event={"ID":"bd1152b0-ccfa-4507-b220-231ee84d3790","Type":"ContainerDied","Data":"13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3"}
Oct 30 01:25:52.503344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.503242    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln" event={"ID":"bd1152b0-ccfa-4507-b220-231ee84d3790","Type":"ContainerDied","Data":"961492def86ccd4978268f732166165a19de5d878ebbf658c09f5788a6987007"}
Oct 30 01:25:52.503344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.503266    6011 scope.go:122] "RemoveContainer" containerID="13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3"
Oct 30 01:25:52.514250 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.514228    6011 scope.go:122] "RemoveContainer" containerID="13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3"
Oct 30 01:25:52.514609 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:52.514588    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3\": container with ID starting with 13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3 not found: ID does not exist" containerID="13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3"
Oct 30 01:25:52.514685 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.514626    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3"} err="failed to get container status \"13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3\": rpc error: code = NotFound desc = could not find container \"13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3\": container with ID starting with 13950d555c60844dd953a71ed9e984ec635e21506da8e4b5d897bf8604b746a3 not found: ID does not exist"
Oct 30 01:25:52.557218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.557110    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frzf4\" (UniqueName: \"kubernetes.io/projected/bd1152b0-ccfa-4507-b220-231ee84d3790-kube-api-access-frzf4\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:52.557218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.557146    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:52.557218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.557163    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd1152b0-ccfa-4507-b220-231ee84d3790-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:52.557478 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.557275    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"]
Oct 30 01:25:52.559895 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:52.559874    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-da76a2-647c49d9f6-bd9ln"]
Oct 30 01:25:54.020530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.020495    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bd1152b0-ccfa-4507-b220-231ee84d3790" path="/var/lib/kubelet/pods/bd1152b0-ccfa-4507-b220-231ee84d3790/volumes"
Oct 30 01:25:54.201047 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.201008    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"]
Oct 30 01:25:54.201364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.201330    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc" podUID="bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" containerName="maas-api" containerID="cri-o://2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c" gracePeriod=30
Oct 30 01:25:54.478332 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.478301    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:25:54.518388 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.518243    6011 generic.go:356] "Generic (PLEG): container finished" podID="bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" containerID="2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c" exitCode=0
Oct 30 01:25:54.518747 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.518350    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"
Oct 30 01:25:54.518855 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.518367    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc" event={"ID":"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d","Type":"ContainerDied","Data":"2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c"}
Oct 30 01:25:54.518902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.518885    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc" event={"ID":"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d","Type":"ContainerDied","Data":"783833ca9156d8851b68d41bc4ecf62bc0b89d21cd8d4768caf40332fc7cdf41"}
Oct 30 01:25:54.518951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.518911    6011 scope.go:122] "RemoveContainer" containerID="2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c"
Oct 30 01:25:54.531447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.531420    6011 scope.go:122] "RemoveContainer" containerID="2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c"
Oct 30 01:25:54.531814 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:54.531789    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c\": container with ID starting with 2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c not found: ID does not exist" containerID="2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c"
Oct 30 01:25:54.531930 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.531823    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c"} err="failed to get container status \"2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c\": rpc error: code = NotFound desc = could not find container \"2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c\": container with ID starting with 2c7a5a945a896179ec410888726ece86a5db10b7e146ba6a708a756f3539c68c not found: ID does not exist"
Oct 30 01:25:54.582075 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.581942    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-metrics-certs\") pod \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") "
Oct 30 01:25:54.582075 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.582067    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-maas-api-tls\") pod \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") "
Oct 30 01:25:54.582502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.582098    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-kube-api-access-hs66w\" (UniqueName: \"kubernetes.io/projected/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-kube-api-access-hs66w\") pod \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") "
Oct 30 01:25:54.582502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.582236    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-openshift-service-ca\") pod \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\" (UID: \"bb35586e-1bee-4d3a-8d7c-d4f66d9f595d\") "
Oct 30 01:25:54.583071 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.583047    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-openshift-service-ca" pod "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" (UID: "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:25:54.585105 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.585078    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-metrics-certs" pod "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" (UID: "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:54.585329 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.585221    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-maas-api-tls" pod "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" (UID: "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:25:54.585329 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.585258    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-kube-api-access-hs66w" pod "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" (UID: "bb35586e-1bee-4d3a-8d7c-d4f66d9f595d"). InnerVolumeSpecName "kube-api-access-hs66w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:25:54.683894 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.683852    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:54.683894 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.683887    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:54.684229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.683914    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs66w\" (UniqueName: \"kubernetes.io/projected/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-kube-api-access-hs66w\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:54.684229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.683926    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:25:54.871080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.870984    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"]
Oct 30 01:25:54.882960 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:54.882926    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-11e7a9-59df47b4cd-bk6lc"]
Oct 30 01:25:56.021975 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:56.021939    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" path="/var/lib/kubelet/pods/bb35586e-1bee-4d3a-8d7c-d4f66d9f595d/volumes"
Oct 30 01:25:57.910620 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:25:57.910580    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:25:58.426101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:58.426066    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"]
Oct 30 01:25:58.426442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:58.426333    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy" containerID="cri-o://349c5523205c67762bb034150aad22a459ed5bb1f591f7a1dd9b2f4224a71782" gracePeriod=30
Oct 30 01:25:59.969799 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:59.969751    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"]
Oct 30 01:25:59.970341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:25:59.970083    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy" containerID="cri-o://6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8" gracePeriod=30
Oct 30 01:26:03.584772 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.584736    6011 generic.go:356] "Generic (PLEG): container finished" podID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerID="349c5523205c67762bb034150aad22a459ed5bb1f591f7a1dd9b2f4224a71782" exitCode=0
Oct 30 01:26:03.585229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.584809    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" event={"ID":"cb6ea9dd-a956-4770-a00c-8dd00370ee41","Type":"ContainerDied","Data":"349c5523205c67762bb034150aad22a459ed5bb1f591f7a1dd9b2f4224a71782"}
Oct 30 01:26:03.711736 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.711672    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 01:26:03.826959 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.826918    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-envoy\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.826959 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.826962    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-certs\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.826981    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-socket\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827012    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-credential-socket\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827082    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-data\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827135    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-podinfo\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827208    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istiod-ca-cert\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827285    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-kube-api-access-kbxkf\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-kube-api-access-kbxkf\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827325    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-token\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-token\") pod \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\" (UID: \"cb6ea9dd-a956-4770-a00c-8dd00370ee41\") "
Oct 30 01:26:03.827562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827349    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-socket" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:03.827562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827363    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-credential-socket" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:03.827562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827458    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-data" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:03.827562 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827309    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-certs" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:03.828046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827646    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istiod-ca-cert" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:26:03.828046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827886    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.828046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827911    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.828046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827927    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.828046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827944    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.828046 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.827958    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.829588 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.829558    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-envoy" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:03.829701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.829621    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-kube-api-access-kbxkf" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "kube-api-access-kbxkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:26:03.829701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.829641    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-podinfo" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:26:03.829701 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.829661    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-token" pod "cb6ea9dd-a956-4770-a00c-8dd00370ee41" (UID: "cb6ea9dd-a956-4770-a00c-8dd00370ee41"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:26:03.929080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.929037    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.929080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.929083    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.929319 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.929100    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbxkf\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-kube-api-access-kbxkf\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:03.929319 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:03.929118    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb6ea9dd-a956-4770-a00c-8dd00370ee41-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:04.592782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:04.592745    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"
Oct 30 01:26:04.592782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:04.592766    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7" event={"ID":"cb6ea9dd-a956-4770-a00c-8dd00370ee41","Type":"ContainerDied","Data":"59553f4538fdfe90ab4586fc978457b88bcd4187cb1f57ccc5f75a6d63f6738a"}
Oct 30 01:26:04.593338 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:04.592845    6011 scope.go:122] "RemoveContainer" containerID="349c5523205c67762bb034150aad22a459ed5bb1f591f7a1dd9b2f4224a71782"
Oct 30 01:26:04.661317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:04.661280    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"]
Oct 30 01:26:04.675036 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:04.675002    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-5a344d-openshift-default-594fb48f99-zzxn7"]
Oct 30 01:26:05.249875 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.249812    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 01:26:05.358877 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.358839    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-kube-api-access-tz7lw\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-kube-api-access-tz7lw\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359054 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.358925    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-credential-socket\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359054 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.358950    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-token\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-token\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359054 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359003    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f18fde74-276f-42ec-a7f7-dda2acb93287-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f18fde74-276f-42ec-a7f7-dda2acb93287-istiod-ca-cert\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359070    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-socket\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359090    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-data\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359154    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-certs\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359196    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-podinfo\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359255 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359223    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-envoy\") pod \"f18fde74-276f-42ec-a7f7-dda2acb93287\" (UID: \"f18fde74-276f-42ec-a7f7-dda2acb93287\") "
Oct 30 01:26:05.359526 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359316    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-credential-socket" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:05.359526 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359408    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-socket" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:05.359618 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359547    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-data" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:05.359618 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359578    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f18fde74-276f-42ec-a7f7-dda2acb93287-istiod-ca-cert" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:26:05.359832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359801    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-certs" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:05.359893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359830    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.359893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359849    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.359893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359858    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.359893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.359867    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f18fde74-276f-42ec-a7f7-dda2acb93287-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.361736 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.361713    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-envoy" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:05.361844 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.361737    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-podinfo" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:26:05.361944 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.361927    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-kube-api-access-tz7lw" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "kube-api-access-tz7lw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:26:05.362003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.361941    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-token" pod "f18fde74-276f-42ec-a7f7-dda2acb93287" (UID: "f18fde74-276f-42ec-a7f7-dda2acb93287"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:26:05.460986 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.460950    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz7lw\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-kube-api-access-tz7lw\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.460986 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.460981    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.460986 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.460991    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.461289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.461000    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.461289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.461007    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f18fde74-276f-42ec-a7f7-dda2acb93287-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:05.601819 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.601781    6011 generic.go:356] "Generic (PLEG): container finished" podID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerID="6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8" exitCode=0
Oct 30 01:26:05.602318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.601832    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"
Oct 30 01:26:05.602318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.601852    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" event={"ID":"f18fde74-276f-42ec-a7f7-dda2acb93287","Type":"ContainerDied","Data":"6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8"}
Oct 30 01:26:05.602318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.601904    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94" event={"ID":"f18fde74-276f-42ec-a7f7-dda2acb93287","Type":"ContainerDied","Data":"b7cd043494d5c68fd98fba5a8bae57f512264bd8bdd2f886cb0e9818041117c0"}
Oct 30 01:26:05.602318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.601934    6011 scope.go:122] "RemoveContainer" containerID="6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8"
Oct 30 01:26:05.612615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.612594    6011 scope.go:122] "RemoveContainer" containerID="6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8"
Oct 30 01:26:05.612931 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:26:05.612908    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8\": container with ID starting with 6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8 not found: ID does not exist" containerID="6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8"
Oct 30 01:26:05.612977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.612940    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8"} err="failed to get container status \"6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8\": rpc error: code = NotFound desc = could not find container \"6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8\": container with ID starting with 6132b6c17ae6b265f5604910b255793c1e3ab04a1eab9c8342e5381d416f7be8 not found: ID does not exist"
Oct 30 01:26:05.656540 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.656502    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"]
Oct 30 01:26:05.661803 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:05.661777    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-da76a2-openshift-default-7f4868c97c-g9f94"]
Oct 30 01:26:06.021351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:06.021262    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" path="/var/lib/kubelet/pods/cb6ea9dd-a956-4770-a00c-8dd00370ee41/volumes"
Oct 30 01:26:06.021765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:06.021750    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" path="/var/lib/kubelet/pods/f18fde74-276f-42ec-a7f7-dda2acb93287/volumes"
Oct 30 01:26:25.556868 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.556831    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"]
Oct 30 01:26:25.557318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.557094    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc" podUID="d3f8731d-4f88-41c3-93a6-66c68bc3f428" containerName="maas-api" containerID="cri-o://876b76a523533bd0adbd2d14c1bca7a3c882eb240c5025d202108d13679a7743" gracePeriod=30
Oct 30 01:26:25.746382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.746346    6011 generic.go:356] "Generic (PLEG): container finished" podID="d3f8731d-4f88-41c3-93a6-66c68bc3f428" containerID="876b76a523533bd0adbd2d14c1bca7a3c882eb240c5025d202108d13679a7743" exitCode=0
Oct 30 01:26:25.746528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.746415    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc" event={"ID":"d3f8731d-4f88-41c3-93a6-66c68bc3f428","Type":"ContainerDied","Data":"876b76a523533bd0adbd2d14c1bca7a3c882eb240c5025d202108d13679a7743"}
Oct 30 01:26:25.846778 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.846745    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:26:25.902307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.899070    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d3f8731d-4f88-41c3-93a6-66c68bc3f428-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f8731d-4f88-41c3-93a6-66c68bc3f428-openshift-service-ca\") pod \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") "
Oct 30 01:26:25.902307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.899237    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-maas-api-tls\") pod \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") "
Oct 30 01:26:25.902307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.899288    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-metrics-certs\") pod \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") "
Oct 30 01:26:25.902307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.899434    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d3f8731d-4f88-41c3-93a6-66c68bc3f428-kube-api-access-hzkkz\" (UniqueName: \"kubernetes.io/projected/d3f8731d-4f88-41c3-93a6-66c68bc3f428-kube-api-access-hzkkz\") pod \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\" (UID: \"d3f8731d-4f88-41c3-93a6-66c68bc3f428\") "
Oct 30 01:26:25.902307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.900619    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3f8731d-4f88-41c3-93a6-66c68bc3f428-openshift-service-ca" pod "d3f8731d-4f88-41c3-93a6-66c68bc3f428" (UID: "d3f8731d-4f88-41c3-93a6-66c68bc3f428"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:26:25.902761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.902714    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3f8731d-4f88-41c3-93a6-66c68bc3f428-kube-api-access-hzkkz" pod "d3f8731d-4f88-41c3-93a6-66c68bc3f428" (UID: "d3f8731d-4f88-41c3-93a6-66c68bc3f428"). InnerVolumeSpecName "kube-api-access-hzkkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:26:25.903061 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.903034    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-maas-api-tls" pod "d3f8731d-4f88-41c3-93a6-66c68bc3f428" (UID: "d3f8731d-4f88-41c3-93a6-66c68bc3f428"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:26:25.941233 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:25.931669    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-metrics-certs" pod "d3f8731d-4f88-41c3-93a6-66c68bc3f428" (UID: "d3f8731d-4f88-41c3-93a6-66c68bc3f428"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:26:26.000985 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.000949    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d3f8731d-4f88-41c3-93a6-66c68bc3f428-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:26.000985 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.000982    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:26.000985 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.000991    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d3f8731d-4f88-41c3-93a6-66c68bc3f428-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:26.001279 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.001003    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzkkz\" (UniqueName: \"kubernetes.io/projected/d3f8731d-4f88-41c3-93a6-66c68bc3f428-kube-api-access-hzkkz\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:26.027338 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.027294    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"]
Oct 30 01:26:26.027595 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.027565    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy" containerID="cri-o://47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5" gracePeriod=30
Oct 30 01:26:26.754447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.754412    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc" event={"ID":"d3f8731d-4f88-41c3-93a6-66c68bc3f428","Type":"ContainerDied","Data":"14c44892c9b5293a027620afa0b6c9a248e3cf3f6d3df2cee72acedfa6fba2bb"}
Oct 30 01:26:26.754447 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.754433    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"
Oct 30 01:26:26.754970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.754498    6011 scope.go:122] "RemoveContainer" containerID="876b76a523533bd0adbd2d14c1bca7a3c882eb240c5025d202108d13679a7743"
Oct 30 01:26:26.807956 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.807920    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"]
Oct 30 01:26:26.821453 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:26.821421    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-cb31e3-c4988bbf7-7nwpc"]
Oct 30 01:26:27.912451 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:26:27.912412    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:26:28.022112 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:28.022074    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d3f8731d-4f88-41c3-93a6-66c68bc3f428" path="/var/lib/kubelet/pods/d3f8731d-4f88-41c3-93a6-66c68bc3f428/volumes"
Oct 30 01:26:31.293758 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.293691    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:26:31.357967 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.357933    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-certs\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.357997    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-envoy\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358045    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-kube-api-access-hwzfz\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-kube-api-access-hwzfz\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358091    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-podinfo\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358348 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358201    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-data\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358348 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358304    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c781a89-e008-4ae5-a88c-8f022004bcc8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c781a89-e008-4ae5-a88c-8f022004bcc8-istiod-ca-cert\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358348    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-socket\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358386    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-token\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-token\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358437    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-credential-socket\") pod \"0c781a89-e008-4ae5-a88c-8f022004bcc8\" (UID: \"0c781a89-e008-4ae5-a88c-8f022004bcc8\") "
Oct 30 01:26:31.358714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358612    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-data" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:31.358773 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.358719    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-certs" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:31.359053 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.359006    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-socket" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:31.359053 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.359011    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-credential-socket" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:31.359287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.359060    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.359287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.359082    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.359287 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.359108    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c781a89-e008-4ae5-a88c-8f022004bcc8-istiod-ca-cert" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:26:31.361000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.360974    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-envoy" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:26:31.361125 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.361049    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-podinfo" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:26:31.361400 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.361375    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-token" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:26:31.361591 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.361573    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-kube-api-access-hwzfz" pod "0c781a89-e008-4ae5-a88c-8f022004bcc8" (UID: "0c781a89-e008-4ae5-a88c-8f022004bcc8"). InnerVolumeSpecName "kube-api-access-hwzfz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:26:31.459881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.459848    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.459881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.459879    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwzfz\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-kube-api-access-hwzfz\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.459881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.459888    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.460130 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.459902    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c781a89-e008-4ae5-a88c-8f022004bcc8-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.460130 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.459915    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.460130 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.459923    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c781a89-e008-4ae5-a88c-8f022004bcc8-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.460130 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.459931    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c781a89-e008-4ae5-a88c-8f022004bcc8-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:26:31.802292 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.802251    6011 generic.go:356] "Generic (PLEG): container finished" podID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerID="47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5" exitCode=0
Oct 30 01:26:31.802489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.802324    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"
Oct 30 01:26:31.802489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.802337    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" event={"ID":"0c781a89-e008-4ae5-a88c-8f022004bcc8","Type":"ContainerDied","Data":"47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5"}
Oct 30 01:26:31.802489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.802429    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7" event={"ID":"0c781a89-e008-4ae5-a88c-8f022004bcc8","Type":"ContainerDied","Data":"bb2f9e2f8a880fed29556cea258db87b60ec79e24b7946e6d574a5945f161ba7"}
Oct 30 01:26:31.802489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.802460    6011 scope.go:122] "RemoveContainer" containerID="47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5"
Oct 30 01:26:31.813693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.813669    6011 scope.go:122] "RemoveContainer" containerID="47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5"
Oct 30 01:26:31.814033 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:26:31.814011    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5\": container with ID starting with 47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5 not found: ID does not exist" containerID="47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5"
Oct 30 01:26:31.814104 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.814046    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5"} err="failed to get container status \"47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5\": rpc error: code = NotFound desc = could not find container \"47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5\": container with ID starting with 47646befcc5aa6347b5603d89b8938b81fc756693468c0d6d887187296be6ef5 not found: ID does not exist"
Oct 30 01:26:31.857016 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.856969    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"]
Oct 30 01:26:31.862667 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:31.862638    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-ad00a0-openshift-default-6bf9d99595-klsm7"]
Oct 30 01:26:32.021961 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:26:32.021927    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" path="/var/lib/kubelet/pods/0c781a89-e008-4ae5-a88c-8f022004bcc8/volumes"
Oct 30 01:26:57.911138 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:26:57.911100    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:27:05.843716 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:05.843668    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"]
Oct 30 01:27:05.844245 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:05.843969    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4" podUID="48b1a7c2-03cc-4809-ad16-7ef76db065fd" containerName="maas-api" containerID="cri-o://1d73ee4d7e8f17a6092a5a2b486ea9f7b4ca6eacdcb152956a6ac59901618309" gracePeriod=30
Oct 30 01:27:06.018339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.018303    6011 generic.go:356] "Generic (PLEG): container finished" podID="48b1a7c2-03cc-4809-ad16-7ef76db065fd" containerID="1d73ee4d7e8f17a6092a5a2b486ea9f7b4ca6eacdcb152956a6ac59901618309" exitCode=0
Oct 30 01:27:06.028882 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.028838    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4" event={"ID":"48b1a7c2-03cc-4809-ad16-7ef76db065fd","Type":"ContainerDied","Data":"1d73ee4d7e8f17a6092a5a2b486ea9f7b4ca6eacdcb152956a6ac59901618309"}
Oct 30 01:27:06.146584 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.146556    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:27:06.285881 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.285847    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-metrics-certs\") pod \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") "
Oct 30 01:27:06.286061 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.285892    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-maas-api-tls\") pod \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") "
Oct 30 01:27:06.286061 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.286034    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48b1a7c2-03cc-4809-ad16-7ef76db065fd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b1a7c2-03cc-4809-ad16-7ef76db065fd-openshift-service-ca\") pod \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") "
Oct 30 01:27:06.286167 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.286066    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48b1a7c2-03cc-4809-ad16-7ef76db065fd-kube-api-access-p46ff\" (UniqueName: \"kubernetes.io/projected/48b1a7c2-03cc-4809-ad16-7ef76db065fd-kube-api-access-p46ff\") pod \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\" (UID: \"48b1a7c2-03cc-4809-ad16-7ef76db065fd\") "
Oct 30 01:27:06.286485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.286447    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48b1a7c2-03cc-4809-ad16-7ef76db065fd-openshift-service-ca" pod "48b1a7c2-03cc-4809-ad16-7ef76db065fd" (UID: "48b1a7c2-03cc-4809-ad16-7ef76db065fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:27:06.286762 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.286738    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/48b1a7c2-03cc-4809-ad16-7ef76db065fd-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:06.288285 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.288254    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-metrics-certs" pod "48b1a7c2-03cc-4809-ad16-7ef76db065fd" (UID: "48b1a7c2-03cc-4809-ad16-7ef76db065fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:27:06.288407 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.288385    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-maas-api-tls" pod "48b1a7c2-03cc-4809-ad16-7ef76db065fd" (UID: "48b1a7c2-03cc-4809-ad16-7ef76db065fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:27:06.288456 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.288434    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48b1a7c2-03cc-4809-ad16-7ef76db065fd-kube-api-access-p46ff" pod "48b1a7c2-03cc-4809-ad16-7ef76db065fd" (UID: "48b1a7c2-03cc-4809-ad16-7ef76db065fd"). InnerVolumeSpecName "kube-api-access-p46ff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:27:06.388115 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.388019    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p46ff\" (UniqueName: \"kubernetes.io/projected/48b1a7c2-03cc-4809-ad16-7ef76db065fd-kube-api-access-p46ff\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:06.388115 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.388050    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:06.388115 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:06.388060    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/48b1a7c2-03cc-4809-ad16-7ef76db065fd-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:07.025229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:07.025194    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"
Oct 30 01:27:07.025715 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:07.025198    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4" event={"ID":"48b1a7c2-03cc-4809-ad16-7ef76db065fd","Type":"ContainerDied","Data":"999aaa5ab90e24a39cd5721ed63042e1caac488d0e2b16b74380dbbeaeacde0d"}
Oct 30 01:27:07.025715 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:07.025340    6011 scope.go:122] "RemoveContainer" containerID="1d73ee4d7e8f17a6092a5a2b486ea9f7b4ca6eacdcb152956a6ac59901618309"
Oct 30 01:27:07.079700 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:07.079660    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"]
Oct 30 01:27:07.092907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:07.092872    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-0519f6-647b54bb5b-mbnj4"]
Oct 30 01:27:08.030003 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:08.029964    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48b1a7c2-03cc-4809-ad16-7ef76db065fd" path="/var/lib/kubelet/pods/48b1a7c2-03cc-4809-ad16-7ef76db065fd/volumes"
Oct 30 01:27:08.711480 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:08.711426    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"]
Oct 30 01:27:08.747489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:08.747454    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"]
Oct 30 01:27:13.712743 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:13.712673    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" podUID="863b2194-53c5-46ba-8f64-6215d4df058b" containerName="payload-processing" containerID="cri-o://7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9" gracePeriod=25
Oct 30 01:27:13.747992 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:13.747946    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" podUID="054a85f3-147d-469f-a4de-a5a29397d439" containerName="payload-pre-processing" containerID="cri-o://1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6" gracePeriod=25
Oct 30 01:27:20.710744 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:20.710705    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" podUID="863b2194-53c5-46ba-8f64-6215d4df058b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:27:20.711152 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:20.710743    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" podUID="054a85f3-147d-469f-a4de-a5a29397d439" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:27:27.910771 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:27:27.910678    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:27:28.232625 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.232524    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"]
Oct 30 01:27:28.232832 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.232800    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw" podUID="10be5e96-2dfe-4713-a2a4-36a42d7c6eba" containerName="maas-api" containerID="cri-o://37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a" gracePeriod=30
Oct 30 01:27:28.506577 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.506550    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 01:27:28.645698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.645666    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-openshift-service-ca\") pod \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") "
Oct 30 01:27:28.645698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.645710    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-kube-api-access-rxqfd\" (UniqueName: \"kubernetes.io/projected/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-kube-api-access-rxqfd\") pod \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") "
Oct 30 01:27:28.645935 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.645818    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-metrics-certs\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-metrics-certs\") pod \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") "
Oct 30 01:27:28.645935 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.645907    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-maas-api-tls\") pod \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\" (UID: \"10be5e96-2dfe-4713-a2a4-36a42d7c6eba\") "
Oct 30 01:27:28.646102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.646072    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-openshift-service-ca" pod "10be5e96-2dfe-4713-a2a4-36a42d7c6eba" (UID: "10be5e96-2dfe-4713-a2a4-36a42d7c6eba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:27:28.646265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.646244    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:28.648169 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.648144    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-maas-api-tls" pod "10be5e96-2dfe-4713-a2a4-36a42d7c6eba" (UID: "10be5e96-2dfe-4713-a2a4-36a42d7c6eba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:27:28.648169 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.648162    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-metrics-certs" pod "10be5e96-2dfe-4713-a2a4-36a42d7c6eba" (UID: "10be5e96-2dfe-4713-a2a4-36a42d7c6eba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:27:28.648351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.648287    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-kube-api-access-rxqfd" pod "10be5e96-2dfe-4713-a2a4-36a42d7c6eba" (UID: "10be5e96-2dfe-4713-a2a4-36a42d7c6eba"). InnerVolumeSpecName "kube-api-access-rxqfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:27:28.747452 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.747360    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:28.747452 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.747405    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:28.747452 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:28.747422    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxqfd\" (UniqueName: \"kubernetes.io/projected/10be5e96-2dfe-4713-a2a4-36a42d7c6eba-kube-api-access-rxqfd\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:29.168647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.168610    6011 generic.go:356] "Generic (PLEG): container finished" podID="10be5e96-2dfe-4713-a2a4-36a42d7c6eba" containerID="37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a" exitCode=0
Oct 30 01:27:29.169106 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.168669    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw" event={"ID":"10be5e96-2dfe-4713-a2a4-36a42d7c6eba","Type":"ContainerDied","Data":"37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a"}
Oct 30 01:27:29.169106 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.168717    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw" event={"ID":"10be5e96-2dfe-4713-a2a4-36a42d7c6eba","Type":"ContainerDied","Data":"236e5ed0cf155e36d6115c82f45c2d7a9f05817c1420e19d5352cab3ee260817"}
Oct 30 01:27:29.169106 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.168673    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"
Oct 30 01:27:29.169106 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.168739    6011 scope.go:122] "RemoveContainer" containerID="37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a"
Oct 30 01:27:29.179809 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.179786    6011 scope.go:122] "RemoveContainer" containerID="37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a"
Oct 30 01:27:29.180140 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:27:29.180119    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a\": container with ID starting with 37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a not found: ID does not exist" containerID="37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a"
Oct 30 01:27:29.180234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.180149    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a"} err="failed to get container status \"37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a\": rpc error: code = NotFound desc = could not find container \"37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a\": container with ID starting with 37e2ee7000961083c3e94e65be2b095d45946bd1649b5986988a579006b4195a not found: ID does not exist"
Oct 30 01:27:29.225826 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.225782    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"]
Oct 30 01:27:29.238906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:29.238871    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-8cf33a-84db68db59-8jsxw"]
Oct 30 01:27:30.021034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:30.021002    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="10be5e96-2dfe-4713-a2a4-36a42d7c6eba" path="/var/lib/kubelet/pods/10be5e96-2dfe-4713-a2a4-36a42d7c6eba/volumes"
Oct 30 01:27:30.711848 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:30.711799    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" podUID="054a85f3-147d-469f-a4de-a5a29397d439" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:27:30.712272 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:30.711852    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" podUID="863b2194-53c5-46ba-8f64-6215d4df058b" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:27:34.027836 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.027788    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 01:27:34.049441 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.049406    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 01:27:34.113296 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.113266    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/863b2194-53c5-46ba-8f64-6215d4df058b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863b2194-53c5-46ba-8f64-6215d4df058b-plugins-config-volume\") pod \"863b2194-53c5-46ba-8f64-6215d4df058b\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") "
Oct 30 01:27:34.113505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.113314    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/863b2194-53c5-46ba-8f64-6215d4df058b-kube-api-access-gxb28\" (UniqueName: \"kubernetes.io/projected/863b2194-53c5-46ba-8f64-6215d4df058b-kube-api-access-gxb28\") pod \"863b2194-53c5-46ba-8f64-6215d4df058b\" (UID: \"863b2194-53c5-46ba-8f64-6215d4df058b\") "
Oct 30 01:27:34.113721 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.113697    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/863b2194-53c5-46ba-8f64-6215d4df058b-plugins-config-volume" pod "863b2194-53c5-46ba-8f64-6215d4df058b" (UID: "863b2194-53c5-46ba-8f64-6215d4df058b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:27:34.115653 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.115627    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/863b2194-53c5-46ba-8f64-6215d4df058b-kube-api-access-gxb28" pod "863b2194-53c5-46ba-8f64-6215d4df058b" (UID: "863b2194-53c5-46ba-8f64-6215d4df058b"). InnerVolumeSpecName "kube-api-access-gxb28". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:27:34.199514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.199472    6011 generic.go:356] "Generic (PLEG): container finished" podID="054a85f3-147d-469f-a4de-a5a29397d439" containerID="1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6" exitCode=0
Oct 30 01:27:34.199514 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.199518    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"
Oct 30 01:27:34.199770 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.199558    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" event={"ID":"054a85f3-147d-469f-a4de-a5a29397d439","Type":"ContainerDied","Data":"1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6"}
Oct 30 01:27:34.199770 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.199616    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl" event={"ID":"054a85f3-147d-469f-a4de-a5a29397d439","Type":"ContainerDied","Data":"462a0ba461cb5893c4bcfd67d8dace707053ff84e90bbd24c869b71479ec3c9a"}
Oct 30 01:27:34.199770 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.199653    6011 scope.go:122] "RemoveContainer" containerID="1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6"
Oct 30 01:27:34.201041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.201020    6011 generic.go:356] "Generic (PLEG): container finished" podID="863b2194-53c5-46ba-8f64-6215d4df058b" containerID="7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9" exitCode=0
Oct 30 01:27:34.201108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.201068    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"
Oct 30 01:27:34.201146 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.201103    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" event={"ID":"863b2194-53c5-46ba-8f64-6215d4df058b","Type":"ContainerDied","Data":"7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9"}
Oct 30 01:27:34.201206 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.201148    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb" event={"ID":"863b2194-53c5-46ba-8f64-6215d4df058b","Type":"ContainerDied","Data":"2a500f4354b25fd412bea6564e9b75fb45d2e5ce93022109942dfd04a52ea085"}
Oct 30 01:27:34.210161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.210145    6011 scope.go:122] "RemoveContainer" containerID="1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6"
Oct 30 01:27:34.210458 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:27:34.210435    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6\": container with ID starting with 1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6 not found: ID does not exist" containerID="1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6"
Oct 30 01:27:34.210559 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.210463    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6"} err="failed to get container status \"1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6\": rpc error: code = NotFound desc = could not find container \"1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6\": container with ID starting with 1c834df390577629d0a1e94f5a8e8d6d90c2f2b63e7812b56e9b4cdac827d5e6 not found: ID does not exist"
Oct 30 01:27:34.210559 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.210480    6011 scope.go:122] "RemoveContainer" containerID="7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9"
Oct 30 01:27:34.214027 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.214006    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/054a85f3-147d-469f-a4de-a5a29397d439-kube-api-access-dgcw7\" (UniqueName: \"kubernetes.io/projected/054a85f3-147d-469f-a4de-a5a29397d439-kube-api-access-dgcw7\") pod \"054a85f3-147d-469f-a4de-a5a29397d439\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") "
Oct 30 01:27:34.214294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.214276    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/054a85f3-147d-469f-a4de-a5a29397d439-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/054a85f3-147d-469f-a4de-a5a29397d439-plugins-config-volume\") pod \"054a85f3-147d-469f-a4de-a5a29397d439\" (UID: \"054a85f3-147d-469f-a4de-a5a29397d439\") "
Oct 30 01:27:34.214633 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.214613    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/863b2194-53c5-46ba-8f64-6215d4df058b-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:34.214720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.214636    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxb28\" (UniqueName: \"kubernetes.io/projected/863b2194-53c5-46ba-8f64-6215d4df058b-kube-api-access-gxb28\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:34.214720 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.214667    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/054a85f3-147d-469f-a4de-a5a29397d439-plugins-config-volume" pod "054a85f3-147d-469f-a4de-a5a29397d439" (UID: "054a85f3-147d-469f-a4de-a5a29397d439"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:27:34.216321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.216299    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/054a85f3-147d-469f-a4de-a5a29397d439-kube-api-access-dgcw7" pod "054a85f3-147d-469f-a4de-a5a29397d439" (UID: "054a85f3-147d-469f-a4de-a5a29397d439"). InnerVolumeSpecName "kube-api-access-dgcw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:27:34.219421 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.219406    6011 scope.go:122] "RemoveContainer" containerID="7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9"
Oct 30 01:27:34.219693 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:27:34.219671    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9\": container with ID starting with 7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9 not found: ID does not exist" containerID="7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9"
Oct 30 01:27:34.219759 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.219705    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9"} err="failed to get container status \"7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9\": rpc error: code = NotFound desc = could not find container \"7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9\": container with ID starting with 7350e758984723adf8a9db41030b76c842e3fdf0fc08b644f05c4862d52670f9 not found: ID does not exist"
Oct 30 01:27:34.254001 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.253924    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"]
Oct 30 01:27:34.257374 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.257347    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w1-8cf33a-7b55dd68b8-25xjb"]
Oct 30 01:27:34.315880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.315848    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/054a85f3-147d-469f-a4de-a5a29397d439-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:34.315880 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.315875    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgcw7\" (UniqueName: \"kubernetes.io/projected/054a85f3-147d-469f-a4de-a5a29397d439-kube-api-access-dgcw7\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:27:34.546734 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.546697    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"]
Oct 30 01:27:34.550521 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:34.550489    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-8cf33a-5c6484f56c-kxzrl"]
Oct 30 01:27:36.020488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:36.020457    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="054a85f3-147d-469f-a4de-a5a29397d439" path="/var/lib/kubelet/pods/054a85f3-147d-469f-a4de-a5a29397d439/volumes"
Oct 30 01:27:36.021040 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:36.021023    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="863b2194-53c5-46ba-8f64-6215d4df058b" path="/var/lib/kubelet/pods/863b2194-53c5-46ba-8f64-6215d4df058b/volumes"
Oct 30 01:27:54.804553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:54.804500    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"]
Oct 30 01:27:54.805168 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:54.804767    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy" containerID="cri-o://6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc" gracePeriod=30
Oct 30 01:27:54.807105 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:54.807078    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"]
Oct 30 01:27:54.807295 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:54.807278    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy" containerID="cri-o://ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff" gracePeriod=30
Oct 30 01:27:54.809532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:54.809506    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"]
Oct 30 01:27:54.809757 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:27:54.809733    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy" containerID="cri-o://2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4" gracePeriod=30
Oct 30 01:27:57.910474 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:27:57.910408    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:28:00.089718 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.089646    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:28:00.117064 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.117038    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:28:00.120575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.120537    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:28:00.202538 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202506    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-envoy\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.202729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202576    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4ca96d71-9252-4b22-9061-de5085b79397-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca96d71-9252-4b22-9061-de5085b79397-istio-podinfo\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.202729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202609    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/136a0179-cd67-4020-8946-60dda0db9b2f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136a0179-cd67-4020-8946-60dda0db9b2f-istio-podinfo\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.202729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202637    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-credential-socket\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.202729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202674    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-istio-token\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-istio-token\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.202729 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202705    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-kube-api-access-9jfcm\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-kube-api-access-9jfcm\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202753    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-podinfo\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202801    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-credential-socket\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202830    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ca96d71-9252-4b22-9061-de5085b79397-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca96d71-9252-4b22-9061-de5085b79397-istiod-ca-cert\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202863    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-credential-socket\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202901    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-data\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202938    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-certs\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202970    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-certs\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.203007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.202968    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-credential-socket" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.203490 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203055    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-data\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.203490 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203261    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-certs" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.203490 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203479    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-credential-socket" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.203662 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203520    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-data" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.203716 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203681    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-data" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.203770 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203743    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-certs" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.203829 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203798    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca96d71-9252-4b22-9061-de5085b79397-istiod-ca-cert" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:00.203887 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203851    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/136a0179-cd67-4020-8946-60dda0db9b2f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136a0179-cd67-4020-8946-60dda0db9b2f-istiod-ca-cert\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.203951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203890    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-istio-token\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-istio-token\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.203951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203923    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2805fa76-0d4d-4185-aacb-b7b5923985a9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2805fa76-0d4d-4185-aacb-b7b5923985a9-istiod-ca-cert\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.204062 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.203955    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-envoy\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.204123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204065    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-credential-socket" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.204123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204098    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-data\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.204253 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204165    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-envoy\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.204253 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204225    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-kube-api-access-cnlnk\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-kube-api-access-cnlnk\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.204530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204306    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-socket\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.204530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204308    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-data" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.204530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204338    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-kube-api-access-57slf\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-kube-api-access-57slf\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.204530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204383    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-token\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-token\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.204530 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204429    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-socket\") pod \"4ca96d71-9252-4b22-9061-de5085b79397\" (UID: \"4ca96d71-9252-4b22-9061-de5085b79397\") "
Oct 30 01:28:00.204823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204535    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-certs\") pod \"2805fa76-0d4d-4185-aacb-b7b5923985a9\" (UID: \"2805fa76-0d4d-4185-aacb-b7b5923985a9\") "
Oct 30 01:28:00.204823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204570    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-socket\") pod \"136a0179-cd67-4020-8946-60dda0db9b2f\" (UID: \"136a0179-cd67-4020-8946-60dda0db9b2f\") "
Oct 30 01:28:00.204823 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.204630    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/136a0179-cd67-4020-8946-60dda0db9b2f-istiod-ca-cert" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:00.205121 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205096    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/136a0179-cd67-4020-8946-60dda0db9b2f-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205126    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205141    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205154    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205167    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ca96d71-9252-4b22-9061-de5085b79397-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205197    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205209    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205222    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205233    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205724 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205246    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.205724 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.205442    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-socket" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.206808 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.206196    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2805fa76-0d4d-4185-aacb-b7b5923985a9-istiod-ca-cert" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:00.206808 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.206257    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-podinfo" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:28:00.206808 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.206639    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-socket" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.206808 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.206778    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-socket" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.207110 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.207022    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-certs" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.208034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.208006    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-envoy" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.208271 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.208222    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/136a0179-cd67-4020-8946-60dda0db9b2f-istio-podinfo" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:28:00.208271 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.208241    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-envoy" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.209343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.209273    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-kube-api-access-9jfcm" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "kube-api-access-9jfcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:00.209343 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.209313    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-istio-token" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:00.209501 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.209354    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-istio-token" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:00.209501 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.209387    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-token" pod "2805fa76-0d4d-4185-aacb-b7b5923985a9" (UID: "2805fa76-0d4d-4185-aacb-b7b5923985a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:00.209620 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.209561    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-kube-api-access-cnlnk" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "kube-api-access-cnlnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:00.210128 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.210095    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-envoy" pod "136a0179-cd67-4020-8946-60dda0db9b2f" (UID: "136a0179-cd67-4020-8946-60dda0db9b2f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:00.211848 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.211817    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-kube-api-access-57slf" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "kube-api-access-57slf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:00.211848 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.211831    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ca96d71-9252-4b22-9061-de5085b79397-istio-podinfo" pod "4ca96d71-9252-4b22-9061-de5085b79397" (UID: "4ca96d71-9252-4b22-9061-de5085b79397"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:28:00.306623 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306583    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306623 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306618    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306623 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306628    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ca96d71-9252-4b22-9061-de5085b79397-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306623 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306636    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/136a0179-cd67-4020-8946-60dda0db9b2f-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306645    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306653    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jfcm\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-kube-api-access-9jfcm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306664    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306672    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306681    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2805fa76-0d4d-4185-aacb-b7b5923985a9-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306688    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/136a0179-cd67-4020-8946-60dda0db9b2f-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306695    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306703    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnlnk\" (UniqueName: \"kubernetes.io/projected/136a0179-cd67-4020-8946-60dda0db9b2f-kube-api-access-cnlnk\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306711    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306719    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57slf\" (UniqueName: \"kubernetes.io/projected/4ca96d71-9252-4b22-9061-de5085b79397-kube-api-access-57slf\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306727    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2805fa76-0d4d-4185-aacb-b7b5923985a9-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306736    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ca96d71-9252-4b22-9061-de5085b79397-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.306908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.306744    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2805fa76-0d4d-4185-aacb-b7b5923985a9-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:00.363813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.363719    6011 generic.go:356] "Generic (PLEG): container finished" podID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerID="ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff" exitCode=0
Oct 30 01:28:00.363813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.363770    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"
Oct 30 01:28:00.363813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.363805    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" event={"ID":"2805fa76-0d4d-4185-aacb-b7b5923985a9","Type":"ContainerDied","Data":"ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff"}
Oct 30 01:28:00.364080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.363863    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88" event={"ID":"2805fa76-0d4d-4185-aacb-b7b5923985a9","Type":"ContainerDied","Data":"02f221209e85e76725f91521bcac79e83a8ae25ae09c3ce749f7e6e0ca829bcd"}
Oct 30 01:28:00.364080 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.363895    6011 scope.go:122] "RemoveContainer" containerID="ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff"
Oct 30 01:28:00.365365 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.365346    6011 generic.go:356] "Generic (PLEG): container finished" podID="136a0179-cd67-4020-8946-60dda0db9b2f" containerID="2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4" exitCode=0
Oct 30 01:28:00.365488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.365376    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" event={"ID":"136a0179-cd67-4020-8946-60dda0db9b2f","Type":"ContainerDied","Data":"2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4"}
Oct 30 01:28:00.365488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.365403    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"
Oct 30 01:28:00.365488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.365425    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb" event={"ID":"136a0179-cd67-4020-8946-60dda0db9b2f","Type":"ContainerDied","Data":"48683abbcafaa51353a3919b7ca50a70404731324b499470422ea5ae1612f744"}
Oct 30 01:28:00.367100 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.367078    6011 generic.go:356] "Generic (PLEG): container finished" podID="4ca96d71-9252-4b22-9061-de5085b79397" containerID="6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc" exitCode=0
Oct 30 01:28:00.367240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.367118    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"
Oct 30 01:28:00.367240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.367153    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" event={"ID":"4ca96d71-9252-4b22-9061-de5085b79397","Type":"ContainerDied","Data":"6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc"}
Oct 30 01:28:00.367240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.367221    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b" event={"ID":"4ca96d71-9252-4b22-9061-de5085b79397","Type":"ContainerDied","Data":"96a2a85414109282a7cc05dd61c1ba1567e0a28431704d8e5b36522897645951"}
Oct 30 01:28:00.376154 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.376133    6011 scope.go:122] "RemoveContainer" containerID="ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff"
Oct 30 01:28:00.376579 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:00.376549    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff\": container with ID starting with ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff not found: ID does not exist" containerID="ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff"
Oct 30 01:28:00.376687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.376583    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff"} err="failed to get container status \"ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff\": rpc error: code = NotFound desc = could not find container \"ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff\": container with ID starting with ba7971fba78565eb091971ae6090f6484eedfc369ab2d98d860b57e18f4610ff not found: ID does not exist"
Oct 30 01:28:00.376687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.376608    6011 scope.go:122] "RemoveContainer" containerID="2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4"
Oct 30 01:28:00.386023 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.386009    6011 scope.go:122] "RemoveContainer" containerID="2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4"
Oct 30 01:28:00.386326 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:00.386306    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4\": container with ID starting with 2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4 not found: ID does not exist" containerID="2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4"
Oct 30 01:28:00.386401 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.386332    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4"} err="failed to get container status \"2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4\": rpc error: code = NotFound desc = could not find container \"2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4\": container with ID starting with 2fee97fd39c4ac9f5ae1d283ed244e402a1faa11d5926c024b4e98feb48a4cc4 not found: ID does not exist"
Oct 30 01:28:00.386401 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.386353    6011 scope.go:122] "RemoveContainer" containerID="6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc"
Oct 30 01:28:00.395794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.395775    6011 scope.go:122] "RemoveContainer" containerID="6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc"
Oct 30 01:28:00.396147 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:00.396097    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc\": container with ID starting with 6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc not found: ID does not exist" containerID="6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc"
Oct 30 01:28:00.396258 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.396146    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc"} err="failed to get container status \"6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc\": rpc error: code = NotFound desc = could not find container \"6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc\": container with ID starting with 6dff645e84d8a7af2061745cd59b457f15c43ff9b41ac8140923cfd68280e3fc not found: ID does not exist"
Oct 30 01:28:00.419127 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.419097    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"]
Oct 30 01:28:00.422470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.422449    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-xqdmb"]
Oct 30 01:28:00.466361 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.466305    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"]
Oct 30 01:28:00.467357 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.467332    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-mlh5b"]
Oct 30 01:28:00.496043 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.496013    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"]
Oct 30 01:28:00.503093 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:00.503060    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w4-60f38f-openshift-default-bc8f76b9c-zct88"]
Oct 30 01:28:02.020963 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:02.020925    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" path="/var/lib/kubelet/pods/136a0179-cd67-4020-8946-60dda0db9b2f/volumes"
Oct 30 01:28:02.021460 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:02.021444    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" path="/var/lib/kubelet/pods/2805fa76-0d4d-4185-aacb-b7b5923985a9/volumes"
Oct 30 01:28:02.022089 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:02.022056    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ca96d71-9252-4b22-9061-de5085b79397" path="/var/lib/kubelet/pods/4ca96d71-9252-4b22-9061-de5085b79397/volumes"
Oct 30 01:28:06.785502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:06.785467    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"]
Oct 30 01:28:06.786021 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:06.785767    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp" podUID="2748e587-acac-4006-a273-e9ddfdd34657" containerName="maas-api" containerID="cri-o://f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5" gracePeriod=30
Oct 30 01:28:06.909935 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:06.909892    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp" podUID="2748e587-acac-4006-a273-e9ddfdd34657" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.143:8443/health\": dial tcp 10.134.0.143:8443: connect: connection refused"
Oct 30 01:28:07.052681 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.052606    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:28:07.178908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.178875    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-metrics-certs\") pod \"2748e587-acac-4006-a273-e9ddfdd34657\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") "
Oct 30 01:28:07.179088 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.178947    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2748e587-acac-4006-a273-e9ddfdd34657-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2748e587-acac-4006-a273-e9ddfdd34657-openshift-service-ca\") pod \"2748e587-acac-4006-a273-e9ddfdd34657\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") "
Oct 30 01:28:07.179088 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.179027    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2748e587-acac-4006-a273-e9ddfdd34657-kube-api-access-dhvtj\" (UniqueName: \"kubernetes.io/projected/2748e587-acac-4006-a273-e9ddfdd34657-kube-api-access-dhvtj\") pod \"2748e587-acac-4006-a273-e9ddfdd34657\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") "
Oct 30 01:28:07.179161 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.179145    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-maas-api-tls\") pod \"2748e587-acac-4006-a273-e9ddfdd34657\" (UID: \"2748e587-acac-4006-a273-e9ddfdd34657\") "
Oct 30 01:28:07.179410 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.179383    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2748e587-acac-4006-a273-e9ddfdd34657-openshift-service-ca" pod "2748e587-acac-4006-a273-e9ddfdd34657" (UID: "2748e587-acac-4006-a273-e9ddfdd34657"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:07.179597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.179576    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2748e587-acac-4006-a273-e9ddfdd34657-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:07.181303 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.181272    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-metrics-certs" pod "2748e587-acac-4006-a273-e9ddfdd34657" (UID: "2748e587-acac-4006-a273-e9ddfdd34657"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:28:07.181413 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.181388    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-maas-api-tls" pod "2748e587-acac-4006-a273-e9ddfdd34657" (UID: "2748e587-acac-4006-a273-e9ddfdd34657"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:28:07.181479 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.181453    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2748e587-acac-4006-a273-e9ddfdd34657-kube-api-access-dhvtj" pod "2748e587-acac-4006-a273-e9ddfdd34657" (UID: "2748e587-acac-4006-a273-e9ddfdd34657"). InnerVolumeSpecName "kube-api-access-dhvtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:07.281078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.281029    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhvtj\" (UniqueName: \"kubernetes.io/projected/2748e587-acac-4006-a273-e9ddfdd34657-kube-api-access-dhvtj\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:07.281078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.281068    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:07.281078 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.281082    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2748e587-acac-4006-a273-e9ddfdd34657-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:07.412311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.412216    6011 generic.go:356] "Generic (PLEG): container finished" podID="2748e587-acac-4006-a273-e9ddfdd34657" containerID="f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5" exitCode=0
Oct 30 01:28:07.412311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.412275    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"
Oct 30 01:28:07.412528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.412304    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp" event={"ID":"2748e587-acac-4006-a273-e9ddfdd34657","Type":"ContainerDied","Data":"f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5"}
Oct 30 01:28:07.412528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.412403    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp" event={"ID":"2748e587-acac-4006-a273-e9ddfdd34657","Type":"ContainerDied","Data":"f5b63b610d9932f62d4f03ce71128ac4130387deb813479f5ff2a1c31ebf9289"}
Oct 30 01:28:07.412528 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.412432    6011 scope.go:122] "RemoveContainer" containerID="f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5"
Oct 30 01:28:07.422601 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.422581    6011 scope.go:122] "RemoveContainer" containerID="f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5"
Oct 30 01:28:07.422910 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:07.422890    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5\": container with ID starting with f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5 not found: ID does not exist" containerID="f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5"
Oct 30 01:28:07.422996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.422953    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5"} err="failed to get container status \"f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5\": rpc error: code = NotFound desc = could not find container \"f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5\": container with ID starting with f58aaad2053ed1b948fa27fbfa5dcc0a0ce4538df8b712cdc1045fdd3d338ef5 not found: ID does not exist"
Oct 30 01:28:07.466451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.466413    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"]
Oct 30 01:28:07.469615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:07.469593    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w4-954bf6-847f55c9d4-wjjfp"]
Oct 30 01:28:08.019692 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:08.019656    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2748e587-acac-4006-a273-e9ddfdd34657" path="/var/lib/kubelet/pods/2748e587-acac-4006-a273-e9ddfdd34657/volumes"
Oct 30 01:28:19.725551 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:19.725462    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"]
Oct 30 01:28:25.356009 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:25.355971    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"]
Oct 30 01:28:25.390408 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:25.390370    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"]
Oct 30 01:28:27.910439 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:27.910405    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:28:30.357336 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:30.357267    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" podUID="af5bdf78-c569-4818-ba87-17d49e069f30" containerName="payload-processing" containerID="cri-o://da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729" gracePeriod=25
Oct 30 01:28:30.391028 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:30.390974    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" podUID="01783257-1c00-4a08-9fc0-af40cf52d2e2" containerName="payload-pre-processing" containerID="cri-o://2d06bbe5c8011712b6744e732f508fcb006fd20f251613f10a9d233fd48c9a58" gracePeriod=25
Oct 30 01:28:34.747372 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:34.747319    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main" containerID="cri-o://e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091" gracePeriod=45
Oct 30 01:28:34.751114 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:34.751075    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" podUID="af5bdf78-c569-4818-ba87-17d49e069f30" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:28:34.751286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:34.751137    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" podUID="01783257-1c00-4a08-9fc0-af40cf52d2e2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:28:34.995337 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:34.995307    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 01:28:35.070513 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.070472    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-home\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-home\") pod \"88a57c40-db65-47d3-a2f3-da243d3caf5a\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") "
Oct 30 01:28:35.070698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.070536    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-dshm\") pod \"88a57c40-db65-47d3-a2f3-da243d3caf5a\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") "
Oct 30 01:28:35.070698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.070608    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/88a57c40-db65-47d3-a2f3-da243d3caf5a-kube-api-access-z2g4g\" (UniqueName: \"kubernetes.io/projected/88a57c40-db65-47d3-a2f3-da243d3caf5a-kube-api-access-z2g4g\") pod \"88a57c40-db65-47d3-a2f3-da243d3caf5a\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") "
Oct 30 01:28:35.070698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.070633    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-tmp-dir\") pod \"88a57c40-db65-47d3-a2f3-da243d3caf5a\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") "
Oct 30 01:28:35.070698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.070663    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/88a57c40-db65-47d3-a2f3-da243d3caf5a-tls-certs\" (UniqueName: \"kubernetes.io/secret/88a57c40-db65-47d3-a2f3-da243d3caf5a-tls-certs\") pod \"88a57c40-db65-47d3-a2f3-da243d3caf5a\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") "
Oct 30 01:28:35.070698 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.070694    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-model-cache\") pod \"88a57c40-db65-47d3-a2f3-da243d3caf5a\" (UID: \"88a57c40-db65-47d3-a2f3-da243d3caf5a\") "
Oct 30 01:28:35.070973 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.070792    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-home" pod "88a57c40-db65-47d3-a2f3-da243d3caf5a" (UID: "88a57c40-db65-47d3-a2f3-da243d3caf5a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:35.071092 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.071060    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-tmp-dir" pod "88a57c40-db65-47d3-a2f3-da243d3caf5a" (UID: "88a57c40-db65-47d3-a2f3-da243d3caf5a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:35.071204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.071124    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:35.071501 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.071455    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-model-cache" pod "88a57c40-db65-47d3-a2f3-da243d3caf5a" (UID: "88a57c40-db65-47d3-a2f3-da243d3caf5a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:35.073455 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.073421    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88a57c40-db65-47d3-a2f3-da243d3caf5a-kube-api-access-z2g4g" pod "88a57c40-db65-47d3-a2f3-da243d3caf5a" (UID: "88a57c40-db65-47d3-a2f3-da243d3caf5a"). InnerVolumeSpecName "kube-api-access-z2g4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:35.073552 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.073479    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-dshm" pod "88a57c40-db65-47d3-a2f3-da243d3caf5a" (UID: "88a57c40-db65-47d3-a2f3-da243d3caf5a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:35.073611 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.073578    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88a57c40-db65-47d3-a2f3-da243d3caf5a-tls-certs" pod "88a57c40-db65-47d3-a2f3-da243d3caf5a" (UID: "88a57c40-db65-47d3-a2f3-da243d3caf5a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:28:35.172553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.172510    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:35.172553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.172553    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2g4g\" (UniqueName: \"kubernetes.io/projected/88a57c40-db65-47d3-a2f3-da243d3caf5a-kube-api-access-z2g4g\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:35.172803 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.172569    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:35.172803 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.172578    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88a57c40-db65-47d3-a2f3-da243d3caf5a-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:35.172803 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.172586    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88a57c40-db65-47d3-a2f3-da243d3caf5a-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:35.575951 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.575914    6011 generic.go:356] "Generic (PLEG): container finished" podID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerID="e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091" exitCode=0
Oct 30 01:28:35.576215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.575971    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"
Oct 30 01:28:35.576215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.576004    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" event={"ID":"88a57c40-db65-47d3-a2f3-da243d3caf5a","Type":"ContainerDied","Data":"e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091"}
Oct 30 01:28:35.576215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.576066    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb" event={"ID":"88a57c40-db65-47d3-a2f3-da243d3caf5a","Type":"ContainerDied","Data":"23d6225d75ce69e3d0b485c1a637b001503a82c20e23b13e4be9d539af2e48ae"}
Oct 30 01:28:35.576215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.576090    6011 scope.go:122] "RemoveContainer" containerID="e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091"
Oct 30 01:28:35.587000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.586976    6011 scope.go:122] "RemoveContainer" containerID="e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091"
Oct 30 01:28:35.587302 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:35.587279    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091\": container with ID starting with e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091 not found: ID does not exist" containerID="e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091"
Oct 30 01:28:35.587390 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.587308    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091"} err="failed to get container status \"e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091\": rpc error: code = NotFound desc = could not find container \"e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091\": container with ID starting with e64862a3b283b8ac0e907ceb027dd86480ff7ef4d41ad18f34a762a4fb63e091 not found: ID does not exist"
Oct 30 01:28:35.629801 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.629760    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"]
Oct 30 01:28:35.643322 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:35.643289    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b565ec/distinct-2-w4-b565ec-kserve-df5d966c8-h4gbb"]
Oct 30 01:28:36.020258 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:36.020157    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" path="/var/lib/kubelet/pods/88a57c40-db65-47d3-a2f3-da243d3caf5a/volumes"
Oct 30 01:28:42.572055 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.572009    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"]
Oct 30 01:28:42.574587 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.572287    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft" podUID="d0dac3d5-db84-4143-8347-ce29d3cf0d2e" containerName="maas-api" containerID="cri-o://8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a" gracePeriod=30
Oct 30 01:28:42.873253 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.867321    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 01:28:42.953099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.953046    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-metrics-certs\") pod \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") "
Oct 30 01:28:42.953440 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.953138    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-kube-api-access-8ll85\" (UniqueName: \"kubernetes.io/projected/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-kube-api-access-8ll85\") pod \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") "
Oct 30 01:28:42.953440 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.953255    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-openshift-service-ca\") pod \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") "
Oct 30 01:28:42.953440 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.953301    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls\") pod \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\" (UID: \"d0dac3d5-db84-4143-8347-ce29d3cf0d2e\") "
Oct 30 01:28:42.953682 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.953648    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-openshift-service-ca" pod "d0dac3d5-db84-4143-8347-ce29d3cf0d2e" (UID: "d0dac3d5-db84-4143-8347-ce29d3cf0d2e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:42.953783 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.953763    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:42.955738 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.955697    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-metrics-certs" pod "d0dac3d5-db84-4143-8347-ce29d3cf0d2e" (UID: "d0dac3d5-db84-4143-8347-ce29d3cf0d2e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:28:42.955864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.955767    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-kube-api-access-8ll85" pod "d0dac3d5-db84-4143-8347-ce29d3cf0d2e" (UID: "d0dac3d5-db84-4143-8347-ce29d3cf0d2e"). InnerVolumeSpecName "kube-api-access-8ll85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:42.955864 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:42.955848    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls" pod "d0dac3d5-db84-4143-8347-ce29d3cf0d2e" (UID: "d0dac3d5-db84-4143-8347-ce29d3cf0d2e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:28:43.054691 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.054646    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8ll85\" (UniqueName: \"kubernetes.io/projected/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-kube-api-access-8ll85\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:43.054691 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.054685    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:43.054691 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.054695    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0dac3d5-db84-4143-8347-ce29d3cf0d2e-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:43.628463 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.628426    6011 generic.go:356] "Generic (PLEG): container finished" podID="d0dac3d5-db84-4143-8347-ce29d3cf0d2e" containerID="8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a" exitCode=0
Oct 30 01:28:43.628907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.628478    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"
Oct 30 01:28:43.628907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.628492    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft" event={"ID":"d0dac3d5-db84-4143-8347-ce29d3cf0d2e","Type":"ContainerDied","Data":"8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a"}
Oct 30 01:28:43.628907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.628536    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft" event={"ID":"d0dac3d5-db84-4143-8347-ce29d3cf0d2e","Type":"ContainerDied","Data":"b9b970db3c77b1268d939a52558a3b8c86acd299791e03f26c3814c56018a0d7"}
Oct 30 01:28:43.628907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.628557    6011 scope.go:122] "RemoveContainer" containerID="8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a"
Oct 30 01:28:43.638774 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.638749    6011 scope.go:122] "RemoveContainer" containerID="8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a"
Oct 30 01:28:43.639058 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:43.639036    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a\": container with ID starting with 8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a not found: ID does not exist" containerID="8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a"
Oct 30 01:28:43.639106 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.639070    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a"} err="failed to get container status \"8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a\": rpc error: code = NotFound desc = could not find container \"8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a\": container with ID starting with 8444ac4189749424579942d1488c2026f94eb5c8c1385cf1a5a4624188fd830a not found: ID does not exist"
Oct 30 01:28:43.685922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.685880    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"]
Oct 30 01:28:43.699316 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:43.699274    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b565ec-89f9d564-5j5ft"]
Oct 30 01:28:44.020043 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:44.019966    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0dac3d5-db84-4143-8347-ce29d3cf0d2e" path="/var/lib/kubelet/pods/d0dac3d5-db84-4143-8347-ce29d3cf0d2e/volumes"
Oct 30 01:28:44.750708 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:44.750668    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" podUID="01783257-1c00-4a08-9fc0-af40cf52d2e2" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:28:44.751249 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:44.750666    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" podUID="af5bdf78-c569-4818-ba87-17d49e069f30" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 01:28:49.596471 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:49.596433    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"]
Oct 30 01:28:49.597000 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:49.596741    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy" containerID="cri-o://d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed" gracePeriod=30
Oct 30 01:28:50.655039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.655008    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 01:28:50.672206 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.672153    6011 generic.go:356] "Generic (PLEG): container finished" podID="af5bdf78-c569-4818-ba87-17d49e069f30" containerID="da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729" exitCode=0
Oct 30 01:28:50.672341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.672231    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" event={"ID":"af5bdf78-c569-4818-ba87-17d49e069f30","Type":"ContainerDied","Data":"da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729"}
Oct 30 01:28:50.672341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.672249    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"
Oct 30 01:28:50.672341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.672304    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx" event={"ID":"af5bdf78-c569-4818-ba87-17d49e069f30","Type":"ContainerDied","Data":"736191b42baf8348c0101afbe2554e40a2e029fe4ac62b447cf12524a15bafcf"}
Oct 30 01:28:50.672341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.672337    6011 scope.go:122] "RemoveContainer" containerID="da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729"
Oct 30 01:28:50.673768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.673734    6011 generic.go:356] "Generic (PLEG): container finished" podID="01783257-1c00-4a08-9fc0-af40cf52d2e2" containerID="2d06bbe5c8011712b6744e732f508fcb006fd20f251613f10a9d233fd48c9a58" exitCode=0
Oct 30 01:28:50.673869 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.673787    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" event={"ID":"01783257-1c00-4a08-9fc0-af40cf52d2e2","Type":"ContainerDied","Data":"2d06bbe5c8011712b6744e732f508fcb006fd20f251613f10a9d233fd48c9a58"}
Oct 30 01:28:50.673869 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.673841    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2" event={"ID":"01783257-1c00-4a08-9fc0-af40cf52d2e2","Type":"ContainerDied","Data":"c3bc5df7bb49217cb65b5d1d563b8e2ceb50a8252a4b7a2f3a7dd2e710af57f3"}
Oct 30 01:28:50.673869 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.673866    6011 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3bc5df7bb49217cb65b5d1d563b8e2ceb50a8252a4b7a2f3a7dd2e710af57f3"
Oct 30 01:28:50.681867 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.681849    6011 scope.go:122] "RemoveContainer" containerID="da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729"
Oct 30 01:28:50.682121 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:50.682102    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729\": container with ID starting with da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729 not found: ID does not exist" containerID="da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729"
Oct 30 01:28:50.682159 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.682133    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729"} err="failed to get container status \"da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729\": rpc error: code = NotFound desc = could not find container \"da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729\": container with ID starting with da961cdd87345d30fd9a01280625c2396b0b68936c0edd7f6e233863aaac8729 not found: ID does not exist"
Oct 30 01:28:50.683424 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.683406    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 01:28:50.727050 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.727011    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af5bdf78-c569-4818-ba87-17d49e069f30-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5bdf78-c569-4818-ba87-17d49e069f30-plugins-config-volume\") pod \"af5bdf78-c569-4818-ba87-17d49e069f30\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") "
Oct 30 01:28:50.727280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.727065    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af5bdf78-c569-4818-ba87-17d49e069f30-kube-api-access-zfcm5\" (UniqueName: \"kubernetes.io/projected/af5bdf78-c569-4818-ba87-17d49e069f30-kube-api-access-zfcm5\") pod \"af5bdf78-c569-4818-ba87-17d49e069f30\" (UID: \"af5bdf78-c569-4818-ba87-17d49e069f30\") "
Oct 30 01:28:50.727280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.727097    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01783257-1c00-4a08-9fc0-af40cf52d2e2-kube-api-access-zsmqw\" (UniqueName: \"kubernetes.io/projected/01783257-1c00-4a08-9fc0-af40cf52d2e2-kube-api-access-zsmqw\") pod \"01783257-1c00-4a08-9fc0-af40cf52d2e2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") "
Oct 30 01:28:50.727280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.727167    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01783257-1c00-4a08-9fc0-af40cf52d2e2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01783257-1c00-4a08-9fc0-af40cf52d2e2-plugins-config-volume\") pod \"01783257-1c00-4a08-9fc0-af40cf52d2e2\" (UID: \"01783257-1c00-4a08-9fc0-af40cf52d2e2\") "
Oct 30 01:28:50.727493 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.727470    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af5bdf78-c569-4818-ba87-17d49e069f30-plugins-config-volume" pod "af5bdf78-c569-4818-ba87-17d49e069f30" (UID: "af5bdf78-c569-4818-ba87-17d49e069f30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:50.727570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.727543    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01783257-1c00-4a08-9fc0-af40cf52d2e2-plugins-config-volume" pod "01783257-1c00-4a08-9fc0-af40cf52d2e2" (UID: "01783257-1c00-4a08-9fc0-af40cf52d2e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:50.729373 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.729349    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af5bdf78-c569-4818-ba87-17d49e069f30-kube-api-access-zfcm5" pod "af5bdf78-c569-4818-ba87-17d49e069f30" (UID: "af5bdf78-c569-4818-ba87-17d49e069f30"). InnerVolumeSpecName "kube-api-access-zfcm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:50.729452 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.729413    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01783257-1c00-4a08-9fc0-af40cf52d2e2-kube-api-access-zsmqw" pod "01783257-1c00-4a08-9fc0-af40cf52d2e2" (UID: "01783257-1c00-4a08-9fc0-af40cf52d2e2"). InnerVolumeSpecName "kube-api-access-zsmqw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:50.828451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.828411    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af5bdf78-c569-4818-ba87-17d49e069f30-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:50.828451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.828447    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfcm5\" (UniqueName: \"kubernetes.io/projected/af5bdf78-c569-4818-ba87-17d49e069f30-kube-api-access-zfcm5\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:50.828451 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.828457    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsmqw\" (UniqueName: \"kubernetes.io/projected/01783257-1c00-4a08-9fc0-af40cf52d2e2-kube-api-access-zsmqw\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:50.828712 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:50.828467    6011 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/01783257-1c00-4a08-9fc0-af40cf52d2e2-plugins-config-volume\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:51.019522 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:51.019485    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"]
Oct 30 01:28:51.024195 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:51.024150    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-b565ec-b965f664b-l86cx"]
Oct 30 01:28:51.680479 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:51.680394    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"
Oct 30 01:28:51.733076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:51.733040    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"]
Oct 30 01:28:51.746268 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:51.746233    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w4-b565ec-75969b8cbc-8szv2"]
Oct 30 01:28:52.020027 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:52.019947    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01783257-1c00-4a08-9fc0-af40cf52d2e2" path="/var/lib/kubelet/pods/01783257-1c00-4a08-9fc0-af40cf52d2e2/volumes"
Oct 30 01:28:52.020367 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:52.020353    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af5bdf78-c569-4818-ba87-17d49e069f30" path="/var/lib/kubelet/pods/af5bdf78-c569-4818-ba87-17d49e069f30/volumes"
Oct 30 01:28:54.856898 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.856874    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 01:28:54.971936 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.971836    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df60db82-e952-4656-a298-6c0cf71e967d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df60db82-e952-4656-a298-6c0cf71e967d-istiod-ca-cert\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972120 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.971968    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-kube-api-access-vnthv\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-kube-api-access-vnthv\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972120 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972009    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-socket\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972120 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972046    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-data\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972120 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972082    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-credential-socket\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972120 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972116    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-envoy\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972463 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972162    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-certs\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972463 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972209    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-istio-token\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-istio-token\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972463 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972309    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/df60db82-e952-4656-a298-6c0cf71e967d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df60db82-e952-4656-a298-6c0cf71e967d-istio-podinfo\") pod \"df60db82-e952-4656-a298-6c0cf71e967d\" (UID: \"df60db82-e952-4656-a298-6c0cf71e967d\") "
Oct 30 01:28:54.972463 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972312    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df60db82-e952-4656-a298-6c0cf71e967d-istiod-ca-cert" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:28:54.972671 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972611    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-certs" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:54.972776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972754    6011 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df60db82-e952-4656-a298-6c0cf71e967d-istiod-ca-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:54.972861 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972783    6011 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:54.972861 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972786    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-credential-socket" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:54.972861 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.972850    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-data" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:54.973213 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.973169    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-socket" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:54.975218 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.975194    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-istio-token" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:54.975380 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.975350    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-envoy" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:28:54.976422 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.976393    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-kube-api-access-vnthv" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "kube-api-access-vnthv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:28:54.989730 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:54.989696    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df60db82-e952-4656-a298-6c0cf71e967d-istio-podinfo" pod "df60db82-e952-4656-a298-6c0cf71e967d" (UID: "df60db82-e952-4656-a298-6c0cf71e967d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 01:28:55.076202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.076141    6011 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-data\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:55.077057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.077029    6011 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-credential-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:55.077289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.077267    6011 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-istio-envoy\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:55.077387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.077294    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:55.077387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.077309    6011 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df60db82-e952-4656-a298-6c0cf71e967d-istio-podinfo\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:55.077387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.077329    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnthv\" (UniqueName: \"kubernetes.io/projected/df60db82-e952-4656-a298-6c0cf71e967d-kube-api-access-vnthv\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:55.077387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.077344    6011 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df60db82-e952-4656-a298-6c0cf71e967d-workload-socket\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:28:55.704931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.704895    6011 generic.go:356] "Generic (PLEG): container finished" podID="df60db82-e952-4656-a298-6c0cf71e967d" containerID="d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed" exitCode=0
Oct 30 01:28:55.705126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.704991    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"
Oct 30 01:28:55.705126 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.704983    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" event={"ID":"df60db82-e952-4656-a298-6c0cf71e967d","Type":"ContainerDied","Data":"d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed"}
Oct 30 01:28:55.705282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.705227    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph" event={"ID":"df60db82-e952-4656-a298-6c0cf71e967d","Type":"ContainerDied","Data":"4446bb13d5b06843316b7c1531fde4a7377ea20f1b129ba15a6ecfc79dbb8283"}
Oct 30 01:28:55.705282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.705267    6011 scope.go:122] "RemoveContainer" containerID="d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed"
Oct 30 01:28:55.716371 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.716354    6011 scope.go:122] "RemoveContainer" containerID="d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed"
Oct 30 01:28:55.716696 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:55.716664    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed\": container with ID starting with d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed not found: ID does not exist" containerID="d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed"
Oct 30 01:28:55.716782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.716703    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed"} err="failed to get container status \"d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed\": rpc error: code = NotFound desc = could not find container \"d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed\": container with ID starting with d817294caea849feebf363c10a0321df72587f9e0ff15e6400bb2cc25ea542ed not found: ID does not exist"
Oct 30 01:28:55.773863 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.773826    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"]
Oct 30 01:28:55.785892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:55.785854    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b565ec-openshift-default-785c49d9bf-8wtph"]
Oct 30 01:28:56.020067 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:28:56.019965    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df60db82-e952-4656-a298-6c0cf71e967d" path="/var/lib/kubelet/pods/df60db82-e952-4656-a298-6c0cf71e967d/volumes"
Oct 30 01:28:57.909664 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:28:57.909631    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:29:19.345462 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:19.345427    6011 scope.go:122] "RemoveContainer" containerID="2d06bbe5c8011712b6744e732f508fcb006fd20f251613f10a9d233fd48c9a58"
Oct 30 01:29:20.795225 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795188    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"]
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795592    6011 cpu_manager.go:414] "containerMap: removing container" podUID="bd1152b0-ccfa-4507-b220-231ee84d3790" containerName="maas-api"
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795604    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bd1152b0-ccfa-4507-b220-231ee84d3790" containerName="maas-api"
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795615    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d3f8731d-4f88-41c3-93a6-66c68bc3f428" containerName="maas-api"
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795620    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d3f8731d-4f88-41c3-93a6-66c68bc3f428" containerName="maas-api"
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795627    6011 cpu_manager.go:414] "containerMap: removing container" podUID="2748e587-acac-4006-a273-e9ddfdd34657" containerName="maas-api"
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795631    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2748e587-acac-4006-a273-e9ddfdd34657" containerName="maas-api"
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795640    6011 cpu_manager.go:414] "containerMap: removing container" podUID="b34b3c21-b0d9-4492-9c42-81ed216616d6" containerName="maas-api"
Oct 30 01:29:20.795647 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795644    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b34b3c21-b0d9-4492-9c42-81ed216616d6" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795658    6011 cpu_manager.go:414] "containerMap: removing container" podUID="af5bdf78-c569-4818-ba87-17d49e069f30" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795663    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af5bdf78-c569-4818-ba87-17d49e069f30" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795668    6011 cpu_manager.go:414] "containerMap: removing container" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795673    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795679    6011 cpu_manager.go:414] "containerMap: removing container" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795683    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795691    6011 cpu_manager.go:414] "containerMap: removing container" podUID="5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795695    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795704    6011 cpu_manager.go:414] "containerMap: removing container" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795708    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795718    6011 cpu_manager.go:414] "containerMap: removing container" podUID="863b2194-53c5-46ba-8f64-6215d4df058b" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795722    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="863b2194-53c5-46ba-8f64-6215d4df058b" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795729    6011 cpu_manager.go:414] "containerMap: removing container" podUID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795733    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795740    6011 cpu_manager.go:414] "containerMap: removing container" podUID="df652682-5a54-4e2e-afdb-25d2dffe9953" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795745    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df652682-5a54-4e2e-afdb-25d2dffe9953" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795754    6011 cpu_manager.go:414] "containerMap: removing container" podUID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795758    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" containerName="payload-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795765    6011 cpu_manager.go:414] "containerMap: removing container" podUID="64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795769    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795778    6011 cpu_manager.go:414] "containerMap: removing container" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795782    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795790    6011 cpu_manager.go:414] "containerMap: removing container" podUID="48b1a7c2-03cc-4809-ad16-7ef76db065fd" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795796    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48b1a7c2-03cc-4809-ad16-7ef76db065fd" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795802    6011 cpu_manager.go:414] "containerMap: removing container" podUID="db66c456-50da-4fa8-9361-24090e0fa74f" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795807    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="db66c456-50da-4fa8-9361-24090e0fa74f" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795815    6011 cpu_manager.go:414] "containerMap: removing container" podUID="c094f8f4-a12e-489e-92b3-908016822212" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795819    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c094f8f4-a12e-489e-92b3-908016822212" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795825    6011 cpu_manager.go:414] "containerMap: removing container" podUID="10be5e96-2dfe-4713-a2a4-36a42d7c6eba" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795829    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="10be5e96-2dfe-4713-a2a4-36a42d7c6eba" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795835    6011 cpu_manager.go:414] "containerMap: removing container" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795839    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795849    6011 cpu_manager.go:414] "containerMap: removing container" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795854    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795861    6011 cpu_manager.go:414] "containerMap: removing container" podUID="01783257-1c00-4a08-9fc0-af40cf52d2e2" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795865    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01783257-1c00-4a08-9fc0-af40cf52d2e2" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795870    6011 cpu_manager.go:414] "containerMap: removing container" podUID="bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795875    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795881    6011 cpu_manager.go:414] "containerMap: removing container" podUID="ee8064a2-6df2-402b-a87b-f10caa63c39a" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795885    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee8064a2-6df2-402b-a87b-f10caa63c39a" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795893    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d0dac3d5-db84-4143-8347-ce29d3cf0d2e" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795898    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0dac3d5-db84-4143-8347-ce29d3cf0d2e" containerName="maas-api"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795904    6011 cpu_manager.go:414] "containerMap: removing container" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795908    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795916    6011 cpu_manager.go:414] "containerMap: removing container" podUID="054a85f3-147d-469f-a4de-a5a29397d439" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795921    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="054a85f3-147d-469f-a4de-a5a29397d439" containerName="payload-pre-processing"
Oct 30 01:29:20.795907 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795929    6011 cpu_manager.go:414] "containerMap: removing container" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795933    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795940    6011 cpu_manager.go:414] "containerMap: removing container" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795944    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795952    6011 cpu_manager.go:414] "containerMap: removing container" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.795956    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796010    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="a05b1685-5535-4c9c-a9cd-cb55f0c79cb0" containerName="payload-processing"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796018    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="df652682-5a54-4e2e-afdb-25d2dffe9953" containerName="payload-pre-processing"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796025    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="a414233b-c7f4-45bf-bb2a-837e6ae8203a" containerName="payload-processing"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796031    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="64ec0ce9-af7f-4d1a-9762-b7f5c55b66bf" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796037    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="136a0179-cd67-4020-8946-60dda0db9b2f" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796043    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="48b1a7c2-03cc-4809-ad16-7ef76db065fd" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796049    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="db66c456-50da-4fa8-9361-24090e0fa74f" containerName="payload-pre-processing"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796054    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="c094f8f4-a12e-489e-92b3-908016822212" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796059    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="10be5e96-2dfe-4713-a2a4-36a42d7c6eba" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796063    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="f18fde74-276f-42ec-a7f7-dda2acb93287" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796069    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c781a89-e008-4ae5-a88c-8f022004bcc8" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796074    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="01783257-1c00-4a08-9fc0-af40cf52d2e2" containerName="payload-pre-processing"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796079    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb35586e-1bee-4d3a-8d7c-d4f66d9f595d" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796083    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee8064a2-6df2-402b-a87b-f10caa63c39a" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796088    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0dac3d5-db84-4143-8347-ce29d3cf0d2e" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796093    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="a922739c-1386-4b81-86f9-4cd0bc393387" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796098    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="054a85f3-147d-469f-a4de-a5a29397d439" containerName="payload-pre-processing"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796103    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="88a57c40-db65-47d3-a2f3-da243d3caf5a" containerName="main"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796107    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="2805fa76-0d4d-4185-aacb-b7b5923985a9" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796113    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="df60db82-e952-4656-a298-6c0cf71e967d" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796119    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="bd1152b0-ccfa-4507-b220-231ee84d3790" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796123    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="d3f8731d-4f88-41c3-93a6-66c68bc3f428" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796130    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="2748e587-acac-4006-a273-e9ddfdd34657" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796136    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="b34b3c21-b0d9-4492-9c42-81ed216616d6" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796141    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="af5bdf78-c569-4818-ba87-17d49e069f30" containerName="payload-processing"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796147    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb6ea9dd-a956-4770-a00c-8dd00370ee41" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796151    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ca96d71-9252-4b22-9061-de5085b79397" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796158    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="5500c7ef-d6bc-4a85-b30d-b57e3e7af43d" containerName="maas-api"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796167    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="9a877407-b50a-41e5-91af-7d9cf4d2fe53" containerName="istio-proxy"
Oct 30 01:29:20.797412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.796190    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="863b2194-53c5-46ba-8f64-6215d4df058b" containerName="payload-processing"
Oct 30 01:29:20.801063 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.801044    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.816960 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.816929    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"]
Oct 30 01:29:20.838433 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838393    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838609 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838461    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3f3f72a-7fe6-4495-a7e8-61f971757191-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838609 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838514    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838611    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838639    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838666    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838727    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838813 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838793    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jm4q\" (UniqueName: \"kubernetes.io/projected/f3f3f72a-7fe6-4495-a7e8-61f971757191-kube-api-access-2jm4q\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.838946 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.838824    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940052    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940099    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940119    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jm4q\" (UniqueName: \"kubernetes.io/projected/f3f3f72a-7fe6-4495-a7e8-61f971757191-kube-api-access-2jm4q\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940149    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940222    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940248    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3f3f72a-7fe6-4495-a7e8-61f971757191-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940287    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940341    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940368    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940691 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940569    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940823    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.940970 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940826    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.941096 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.940973    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.941170 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.941150    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f3f3f72a-7fe6-4495-a7e8-61f971757191-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.942723 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.942694    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.942900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.942882    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.962101 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.962069    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f3f3f72a-7fe6-4495-a7e8-61f971757191-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:20.962318 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:20.962296    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jm4q\" (UniqueName: \"kubernetes.io/projected/f3f3f72a-7fe6-4495-a7e8-61f971757191-kube-api-access-2jm4q\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4dwdf\" (UID: \"f3f3f72a-7fe6-4495-a7e8-61f971757191\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:21.116271 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.116147    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:21.433062 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.433026    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"]
Oct 30 01:29:21.433981 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:29:21.433957    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3f3f72a_7fe6_4495_a7e8_61f971757191.slice/crio-4afcc08343bfb0a3cd9f1c63ea00cd33f42bee7900ecc5867028677eb5155c2a WatchSource:0}: Error finding container 4afcc08343bfb0a3cd9f1c63ea00cd33f42bee7900ecc5867028677eb5155c2a: Status 404 returned error can't find the container with id 4afcc08343bfb0a3cd9f1c63ea00cd33f42bee7900ecc5867028677eb5155c2a
Oct 30 01:29:21.436164 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.436131    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:29:21.436259 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.436241    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:29:21.436311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.436279    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:29:21.853937 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.853738    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" event={"ID":"f3f3f72a-7fe6-4495-a7e8-61f971757191","Type":"ContainerStarted","Data":"ff131a2cefb7fff836e0616289169c8dd372f478582f9ec226dce589f3e21e63"}
Oct 30 01:29:21.853937 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.853815    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" event={"ID":"f3f3f72a-7fe6-4495-a7e8-61f971757191","Type":"ContainerStarted","Data":"4afcc08343bfb0a3cd9f1c63ea00cd33f42bee7900ecc5867028677eb5155c2a"}
Oct 30 01:29:21.926382 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:21.926319    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podStartSLOduration=1.926300999 podStartE2EDuration="1.926300999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:29:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:29:21.911166051 +0000 UTC m=+3004.013920022" watchObservedRunningTime="2026-10-30 01:29:21.926300999 +0000 UTC m=+3004.029054964"
Oct 30 01:29:22.117037 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:22.116950    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:22.118281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:22.118257    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 01:29:22.118346 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:22.118309    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 01:29:23.116889 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:23.116859    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 01:29:23.117293 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:23.116920    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 01:29:24.116597 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:24.116552    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 01:29:24.116781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:24.116615    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 01:29:25.117122 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:25.117084    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 01:29:25.117546 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:25.117139    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 01:29:26.117288 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:26.117250    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 01:29:26.117703 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:26.117310    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 01:29:27.117387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:27.117348    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 01:29:27.117804 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:27.117405    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 01:29:27.911214 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:29:27.911165    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:29:28.116773 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:28.116737    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused" start-of-body=
Oct 30 01:29:28.116949 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:28.116798    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": dial tcp 10.134.0.157:15021: connect: connection refused"
Oct 30 01:29:29.769479 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:29.769431    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:29.769974 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:29.769802    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:29.770990 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:29:29.770965    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf"
Oct 30 01:29:57.911198 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:29:57.911146    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:30:00.812399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:00.812339    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4dwdf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:30:00.812978 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:00.812420    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4dwdf" podUID="f3f3f72a-7fe6-4495-a7e8-61f971757191" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.157:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:30:20.783539 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.783492    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"]
Oct 30 01:30:20.787605 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.787570    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.821276 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.821241    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"]
Oct 30 01:30:20.860410 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.860277    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.860628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.860440    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.860628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.860484    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.860628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.860519    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.860628 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.860577    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a730d256-143d-4ee5-91ed-e099a288a7f5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.860999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.860605    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w85gj\" (UniqueName: \"kubernetes.io/projected/a730d256-143d-4ee5-91ed-e099a288a7f5-kube-api-access-w85gj\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.861065 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.861013    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.861142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.861121    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.861415 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.861380    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962321 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962276    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962334    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a730d256-143d-4ee5-91ed-e099a288a7f5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962354    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w85gj\" (UniqueName: \"kubernetes.io/projected/a730d256-143d-4ee5-91ed-e099a288a7f5-kube-api-access-w85gj\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962392    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962474    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962499    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962553 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962540    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962904 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962635    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962904 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962664    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.962904 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962866    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.963070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962917    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.963070 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.962961    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.963254 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.963209    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.963776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.963747    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a730d256-143d-4ee5-91ed-e099a288a7f5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.965311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.965283    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.965417 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.965398    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.982763 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.982724    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a730d256-143d-4ee5-91ed-e099a288a7f5-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:20.982892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:20.982865    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w85gj\" (UniqueName: \"kubernetes.io/projected/a730d256-143d-4ee5-91ed-e099a288a7f5-kube-api-access-w85gj\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nnnk8\" (UID: \"a730d256-143d-4ee5-91ed-e099a288a7f5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:21.101572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:21.101474    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:21.427261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:21.427229    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"]
Oct 30 01:30:21.428523 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:30:21.428476    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda730d256_143d_4ee5_91ed_e099a288a7f5.slice/crio-2ac356b92d17fa9b200c835e5a983240dd6a9cd76cbdf7b28ae8ca2360a90c11 WatchSource:0}: Error finding container 2ac356b92d17fa9b200c835e5a983240dd6a9cd76cbdf7b28ae8ca2360a90c11: Status 404 returned error can't find the container with id 2ac356b92d17fa9b200c835e5a983240dd6a9cd76cbdf7b28ae8ca2360a90c11
Oct 30 01:30:21.431507 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:21.431453    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:30:21.431604 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:21.431552    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:30:21.431659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:21.431599    6011 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892008Ki","pods":"250"}
Oct 30 01:30:22.224029 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:22.223985    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" event={"ID":"a730d256-143d-4ee5-91ed-e099a288a7f5","Type":"ContainerStarted","Data":"2817d84ce97bd5641a6fa0d9e955ca186854ba5cb9b8f9202307cbf38b3372a0"}
Oct 30 01:30:22.224607 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:22.224051    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" event={"ID":"a730d256-143d-4ee5-91ed-e099a288a7f5","Type":"ContainerStarted","Data":"2ac356b92d17fa9b200c835e5a983240dd6a9cd76cbdf7b28ae8ca2360a90c11"}
Oct 30 01:30:22.317203 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:22.317020    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" podStartSLOduration=2.31699925 podStartE2EDuration="2.31699925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:30:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:30:22.298857562 +0000 UTC m=+3064.401611562" watchObservedRunningTime="2026-10-30 01:30:22.31699925 +0000 UTC m=+3064.419753222"
Oct 30 01:30:23.109204 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:23.108493    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:23.110268 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:23.109938    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nnnk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 01:30:23.110268 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:23.110094    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" podUID="a730d256-143d-4ee5-91ed-e099a288a7f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 01:30:24.102492 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:24.102456    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nnnk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 01:30:24.102924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:24.102508    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" podUID="a730d256-143d-4ee5-91ed-e099a288a7f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 01:30:25.101901 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:25.101859    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nnnk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 01:30:25.102105 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:25.101931    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" podUID="a730d256-143d-4ee5-91ed-e099a288a7f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 01:30:26.102280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:26.102243    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nnnk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 01:30:26.102741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:26.102311    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" podUID="a730d256-143d-4ee5-91ed-e099a288a7f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 01:30:27.102286 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:27.102248    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nnnk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 01:30:27.102482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:27.102316    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" podUID="a730d256-143d-4ee5-91ed-e099a288a7f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 01:30:27.910241 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:30:27.910204    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:30:28.102669 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:28.102623    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nnnk8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused" start-of-body=
Oct 30 01:30:28.103108 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:28.102713    6011 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8" podUID="a730d256-143d-4ee5-91ed-e099a288a7f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.158:15021/healthz/ready\": dial tcp 10.134.0.158:15021: connect: connection refused"
Oct 30 01:30:29.847767 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:29.847727    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:29.848448 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:29.848247    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:29.849245 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:30:29.849221    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nnnk8"
Oct 30 01:30:57.910084 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:30:57.910042    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:31:20.856356 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:20.856318    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"]
Oct 30 01:31:20.856945 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:20.856537    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj" podUID="d16e2564-d89c-4ecf-95fa-b41fe949f901" containerName="discovery" containerID="cri-o://bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e" gracePeriod=30
Oct 30 01:31:21.116795 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.116733    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 01:31:21.209381 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.209347    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-kubeconfig\") pod \"d16e2564-d89c-4ecf-95fa-b41fe949f901\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") "
Oct 30 01:31:21.209570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.209411    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-kube-api-access-g87qw\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-kube-api-access-g87qw\") pod \"d16e2564-d89c-4ecf-95fa-b41fe949f901\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") "
Oct 30 01:31:21.209570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.209493    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-dns-cert\") pod \"d16e2564-d89c-4ecf-95fa-b41fe949f901\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") "
Oct 30 01:31:21.209570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.209543    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-token\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-token\") pod \"d16e2564-d89c-4ecf-95fa-b41fe949f901\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") "
Oct 30 01:31:21.209570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.209567    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d16e2564-d89c-4ecf-95fa-b41fe949f901-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16e2564-d89c-4ecf-95fa-b41fe949f901-local-certs\") pod \"d16e2564-d89c-4ecf-95fa-b41fe949f901\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") "
Oct 30 01:31:21.209814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.209596    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-cacerts\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-cacerts\") pod \"d16e2564-d89c-4ecf-95fa-b41fe949f901\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") "
Oct 30 01:31:21.209814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.209646    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-ca-configmap\") pod \"d16e2564-d89c-4ecf-95fa-b41fe949f901\" (UID: \"d16e2564-d89c-4ecf-95fa-b41fe949f901\") "
Oct 30 01:31:21.210507 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.210469    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-ca-configmap" pod "d16e2564-d89c-4ecf-95fa-b41fe949f901" (UID: "d16e2564-d89c-4ecf-95fa-b41fe949f901"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:31:21.211904 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.211863    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-dns-cert" pod "d16e2564-d89c-4ecf-95fa-b41fe949f901" (UID: "d16e2564-d89c-4ecf-95fa-b41fe949f901"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:31:21.212034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.211984    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d16e2564-d89c-4ecf-95fa-b41fe949f901-local-certs" pod "d16e2564-d89c-4ecf-95fa-b41fe949f901" (UID: "d16e2564-d89c-4ecf-95fa-b41fe949f901"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:31:21.212433 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.212409    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-kubeconfig" pod "d16e2564-d89c-4ecf-95fa-b41fe949f901" (UID: "d16e2564-d89c-4ecf-95fa-b41fe949f901"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:31:21.212498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.212434    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-token" pod "d16e2564-d89c-4ecf-95fa-b41fe949f901" (UID: "d16e2564-d89c-4ecf-95fa-b41fe949f901"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:31:21.212498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.212436    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-cacerts" pod "d16e2564-d89c-4ecf-95fa-b41fe949f901" (UID: "d16e2564-d89c-4ecf-95fa-b41fe949f901"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:31:21.212498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.212448    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-kube-api-access-g87qw" pod "d16e2564-d89c-4ecf-95fa-b41fe949f901" (UID: "d16e2564-d89c-4ecf-95fa-b41fe949f901"). InnerVolumeSpecName "kube-api-access-g87qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:31:21.311482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.311449    6011 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-token\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:31:21.311482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.311480    6011 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d16e2564-d89c-4ecf-95fa-b41fe949f901-local-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:31:21.311482 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.311490    6011 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-cacerts\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:31:21.311735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.311499    6011 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-ca-configmap\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:31:21.311735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.311509    6011 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-kubeconfig\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:31:21.311735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.311518    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g87qw\" (UniqueName: \"kubernetes.io/projected/d16e2564-d89c-4ecf-95fa-b41fe949f901-kube-api-access-g87qw\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:31:21.311735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.311526    6011 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d16e2564-d89c-4ecf-95fa-b41fe949f901-istio-csr-dns-cert\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:31:21.645300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.645263    6011 generic.go:356] "Generic (PLEG): container finished" podID="d16e2564-d89c-4ecf-95fa-b41fe949f901" containerID="bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e" exitCode=0
Oct 30 01:31:21.645503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.645320    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"
Oct 30 01:31:21.645503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.645351    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj" event={"ID":"d16e2564-d89c-4ecf-95fa-b41fe949f901","Type":"ContainerDied","Data":"bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e"}
Oct 30 01:31:21.645503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.645406    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj" event={"ID":"d16e2564-d89c-4ecf-95fa-b41fe949f901","Type":"ContainerDied","Data":"98b7a6b1e70511d36252166709e52c35b8c0d9183c1887bf0870a4055b83a281"}
Oct 30 01:31:21.645503 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.645428    6011 scope.go:122] "RemoveContainer" containerID="bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e"
Oct 30 01:31:21.657019 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.656996    6011 scope.go:122] "RemoveContainer" containerID="bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e"
Oct 30 01:31:21.657328 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:31:21.657307    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e\": container with ID starting with bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e not found: ID does not exist" containerID="bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e"
Oct 30 01:31:21.657387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.657340    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e"} err="failed to get container status \"bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e\": rpc error: code = NotFound desc = could not find container \"bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e\": container with ID starting with bfa3c31feb00f167358da9ba97db8dcb5df0feef32d74b48cabb4927dd12eb6e not found: ID does not exist"
Oct 30 01:31:21.703709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.703674    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"]
Oct 30 01:31:21.717053 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:21.717014    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-nvwbj"]
Oct 30 01:31:22.019659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:31:22.019579    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d16e2564-d89c-4ecf-95fa-b41fe949f901" path="/var/lib/kubelet/pods/d16e2564-d89c-4ecf-95fa-b41fe949f901/volumes"
Oct 30 01:31:27.909224 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:31:27.909168    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:31:57.908872 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:31:57.908824    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:32:27.910353 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:32:27.910307    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:32:57.910395 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:32:57.910354    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:33:27.910420 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:33:27.910330    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:33:57.910296 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:33:57.910252    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:33:59.627111 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.627076    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"]
Oct 30 01:33:59.628153 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.627703    6011 cpu_manager.go:414] "containerMap: removing container" podUID="d16e2564-d89c-4ecf-95fa-b41fe949f901" containerName="discovery"
Oct 30 01:33:59.628153 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.627718    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d16e2564-d89c-4ecf-95fa-b41fe949f901" containerName="discovery"
Oct 30 01:33:59.628153 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.627810    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="d16e2564-d89c-4ecf-95fa-b41fe949f901" containerName="discovery"
Oct 30 01:33:59.631197 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.631154    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.661722 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.661688    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-364d04-serving-cert\""
Oct 30 01:33:59.662964 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.662920    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-metrics-certs\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.663128 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.662972    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b51c06-3032-4203-804f-0a8b95bc1daa-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.663128 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.663110    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkppl\" (UniqueName: \"kubernetes.io/projected/57b51c06-3032-4203-804f-0a8b95bc1daa-kube-api-access-rkppl\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.663300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.663218    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.663984 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.663959    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"]
Oct 30 01:33:59.764613 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.764577    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkppl\" (UniqueName: \"kubernetes.io/projected/57b51c06-3032-4203-804f-0a8b95bc1daa-kube-api-access-rkppl\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.764805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.764638    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.764805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.764693    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-metrics-certs\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.764805 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.764711    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b51c06-3032-4203-804f-0a8b95bc1daa-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.765372 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.765350    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b51c06-3032-4203-804f-0a8b95bc1daa-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.767581 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.767554    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-metrics-certs\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.767581 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.767574    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.789282 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.789245    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkppl\" (UniqueName: \"kubernetes.io/projected/57b51c06-3032-4203-804f-0a8b95bc1daa-kube-api-access-rkppl\") pod \"maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:33:59.945549 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:33:59.945459    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:34:00.244428 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:00.244401    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"]
Oct 30 01:34:00.247026 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:34:00.246994    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57b51c06_3032_4203_804f_0a8b95bc1daa.slice/crio-9c7868fb0c3fd4ee9e5054e09d50ad3dc6e78668750afae07f47a5ed4f1d33a9 WatchSource:0}: Error finding container 9c7868fb0c3fd4ee9e5054e09d50ad3dc6e78668750afae07f47a5ed4f1d33a9: Status 404 returned error can't find the container with id 9c7868fb0c3fd4ee9e5054e09d50ad3dc6e78668750afae07f47a5ed4f1d33a9
Oct 30 01:34:00.248491 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:00.248473    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:34:00.619920 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:00.619882    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4" event={"ID":"57b51c06-3032-4203-804f-0a8b95bc1daa","Type":"ContainerStarted","Data":"9c7868fb0c3fd4ee9e5054e09d50ad3dc6e78668750afae07f47a5ed4f1d33a9"}
Oct 30 01:34:01.626924 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:01.626884    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4" event={"ID":"57b51c06-3032-4203-804f-0a8b95bc1daa","Type":"ContainerStarted","Data":"26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d"}
Oct 30 01:34:01.627369 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:01.626947    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:34:01.669948 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:01.669897    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4" podStartSLOduration=2.341433625 podStartE2EDuration="2.669882783s" totalImagesPullingTime="328.449158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:34:01.668355699 +0000 UTC m=+3283.771109670" watchObservedRunningTime="2026-10-30 01:34:01.669882783 +0000 UTC m=+3283.772636799"
Oct 30 01:34:07.638226 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:07.638170    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:34:15.082116 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.082067    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"]
Oct 30 01:34:15.086385 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.086362    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.104434 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.104406    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"]
Oct 30 01:34:15.115771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.115741    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-364d04\"/\"openshift-service-ca.crt\""
Oct 30 01:34:15.115771 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.115769    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-364d04\"/\"kube-root-ca.crt\""
Oct 30 01:34:15.179714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.179675    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-364d04\"/\"default-dockercfg-v8kfd\""
Oct 30 01:34:15.179714 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.179704    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-364d04\"/\"simulator-single-main-364d04-kserve-self-signed-certs\""
Oct 30 01:34:15.231629 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.231596    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-tmp-dir\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.231844 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.231644    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svkn6\" (UniqueName: \"kubernetes.io/projected/71d572c2-09da-4321-bdd8-f5f9cc71891a-kube-api-access-svkn6\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.231844 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.231667    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d572c2-09da-4321-bdd8-f5f9cc71891a-tls-certs\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.231844 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.231810    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-model-cache\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.232026 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.231883    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-dshm\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.232089 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.232062    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-home\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.332958 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.332869    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-home\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.332958 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.332918    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-tmp-dir\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.332958 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.332946    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svkn6\" (UniqueName: \"kubernetes.io/projected/71d572c2-09da-4321-bdd8-f5f9cc71891a-kube-api-access-svkn6\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.333300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.332966    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d572c2-09da-4321-bdd8-f5f9cc71891a-tls-certs\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.333300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.332990    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-model-cache\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.333300 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.333011    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-dshm\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.333470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.333382    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-home\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.333470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.333391    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-model-cache\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.333470 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.333433    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-tmp-dir\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.335601 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.335580    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-dshm\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.335719 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.335698    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d572c2-09da-4321-bdd8-f5f9cc71891a-tls-certs\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.353132 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.353102    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svkn6\" (UniqueName: \"kubernetes.io/projected/71d572c2-09da-4321-bdd8-f5f9cc71891a-kube-api-access-svkn6\") pod \"simulator-single-main-364d04-kserve-558f95d7f5-qgvwm\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") " pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.408404 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.408369    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:15.705489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:15.705463    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"]
Oct 30 01:34:15.706632 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:34:15.706603    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71d572c2_09da_4321_bdd8_f5f9cc71891a.slice/crio-253f37f30e3f4944d5a08e83cd3d4510f55ed924e90b5670439a6b5827a8ff55 WatchSource:0}: Error finding container 253f37f30e3f4944d5a08e83cd3d4510f55ed924e90b5670439a6b5827a8ff55: Status 404 returned error can't find the container with id 253f37f30e3f4944d5a08e83cd3d4510f55ed924e90b5670439a6b5827a8ff55
Oct 30 01:34:16.711301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:16.711263    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm" event={"ID":"71d572c2-09da-4321-bdd8-f5f9cc71891a","Type":"ContainerStarted","Data":"d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c"}
Oct 30 01:34:16.711301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:16.711315    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm" event={"ID":"71d572c2-09da-4321-bdd8-f5f9cc71891a","Type":"ContainerStarted","Data":"253f37f30e3f4944d5a08e83cd3d4510f55ed924e90b5670439a6b5827a8ff55"}
Oct 30 01:34:16.752670 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:16.752626    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm" podStartSLOduration=1.752613133 podStartE2EDuration="1.752613133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:34:16.751988453 +0000 UTC m=+3298.854742423" watchObservedRunningTime="2026-10-30 01:34:16.752613133 +0000 UTC m=+3298.855367097"
Oct 30 01:34:17.408931 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:17.408900    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:25.408591 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:25.408545    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:25.421378 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:25.421354    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:25.775021 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:25.774939    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:34:27.909531 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:34:27.909494    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:34:54.457983 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:34:54.457949    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"]
Oct 30 01:34:57.910030 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:34:57.909988    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:35:01.887560 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:01.887512    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"]
Oct 30 01:35:01.888075 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:01.887798    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4" podUID="57b51c06-3032-4203-804f-0a8b95bc1daa" containerName="maas-api" containerID="cri-o://26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d" gracePeriod=30
Oct 30 01:35:02.345122 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.345093    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:35:02.435307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.435204    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57b51c06-3032-4203-804f-0a8b95bc1daa-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b51c06-3032-4203-804f-0a8b95bc1daa-openshift-service-ca\") pod \"57b51c06-3032-4203-804f-0a8b95bc1daa\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") "
Oct 30 01:35:02.435307 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.435297    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-metrics-certs\") pod \"57b51c06-3032-4203-804f-0a8b95bc1daa\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") "
Oct 30 01:35:02.435539 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.435318    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57b51c06-3032-4203-804f-0a8b95bc1daa-kube-api-access-rkppl\" (UniqueName: \"kubernetes.io/projected/57b51c06-3032-4203-804f-0a8b95bc1daa-kube-api-access-rkppl\") pod \"57b51c06-3032-4203-804f-0a8b95bc1daa\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") "
Oct 30 01:35:02.435539 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.435432    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-maas-api-tls\") pod \"57b51c06-3032-4203-804f-0a8b95bc1daa\" (UID: \"57b51c06-3032-4203-804f-0a8b95bc1daa\") "
Oct 30 01:35:02.435661 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.435633    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57b51c06-3032-4203-804f-0a8b95bc1daa-openshift-service-ca" pod "57b51c06-3032-4203-804f-0a8b95bc1daa" (UID: "57b51c06-3032-4203-804f-0a8b95bc1daa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:35:02.435827 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.435809    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/57b51c06-3032-4203-804f-0a8b95bc1daa-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:02.437659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.437626    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57b51c06-3032-4203-804f-0a8b95bc1daa-kube-api-access-rkppl" pod "57b51c06-3032-4203-804f-0a8b95bc1daa" (UID: "57b51c06-3032-4203-804f-0a8b95bc1daa"). InnerVolumeSpecName "kube-api-access-rkppl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:35:02.437659 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.437632    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-metrics-certs" pod "57b51c06-3032-4203-804f-0a8b95bc1daa" (UID: "57b51c06-3032-4203-804f-0a8b95bc1daa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:35:02.437810 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.437702    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-maas-api-tls" pod "57b51c06-3032-4203-804f-0a8b95bc1daa" (UID: "57b51c06-3032-4203-804f-0a8b95bc1daa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:35:02.536838 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.536795    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:02.536838 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.536835    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/57b51c06-3032-4203-804f-0a8b95bc1daa-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:02.536838 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.536845    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkppl\" (UniqueName: \"kubernetes.io/projected/57b51c06-3032-4203-804f-0a8b95bc1daa-kube-api-access-rkppl\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:02.964776 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.964741    6011 generic.go:356] "Generic (PLEG): container finished" podID="57b51c06-3032-4203-804f-0a8b95bc1daa" containerID="26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d" exitCode=0
Oct 30 01:35:02.965261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.964788    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"
Oct 30 01:35:02.965261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.964833    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4" event={"ID":"57b51c06-3032-4203-804f-0a8b95bc1daa","Type":"ContainerDied","Data":"26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d"}
Oct 30 01:35:02.965261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.964885    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4" event={"ID":"57b51c06-3032-4203-804f-0a8b95bc1daa","Type":"ContainerDied","Data":"9c7868fb0c3fd4ee9e5054e09d50ad3dc6e78668750afae07f47a5ed4f1d33a9"}
Oct 30 01:35:02.965261 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.964909    6011 scope.go:122] "RemoveContainer" containerID="26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d"
Oct 30 01:35:02.975889 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.975866    6011 scope.go:122] "RemoveContainer" containerID="26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d"
Oct 30 01:35:02.976276 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:35:02.976255    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d\": container with ID starting with 26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d not found: ID does not exist" containerID="26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d"
Oct 30 01:35:02.976351 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:02.976289    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d"} err="failed to get container status \"26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d\": rpc error: code = NotFound desc = could not find container \"26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d\": container with ID starting with 26f29ea4575349a101edebbd13204a27de19129f55ec38537496c055f141221d not found: ID does not exist"
Oct 30 01:35:03.019955 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:03.019921    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"]
Oct 30 01:35:03.033613 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:03.033579    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-364d04-58b6bcb8c6-xfhn4"]
Oct 30 01:35:04.020923 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:04.020892    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57b51c06-3032-4203-804f-0a8b95bc1daa" path="/var/lib/kubelet/pods/57b51c06-3032-4203-804f-0a8b95bc1daa/volumes"
Oct 30 01:35:09.478906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.478848    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm" podUID="71d572c2-09da-4321-bdd8-f5f9cc71891a" containerName="main" containerID="cri-o://d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c" gracePeriod=45
Oct 30 01:35:09.744405 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.744337    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:35:09.809893 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.809858    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-home\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-home\") pod \"71d572c2-09da-4321-bdd8-f5f9cc71891a\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") "
Oct 30 01:35:09.810095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.809914    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-dshm\") pod \"71d572c2-09da-4321-bdd8-f5f9cc71891a\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") "
Oct 30 01:35:09.810095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.809960    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-model-cache\") pod \"71d572c2-09da-4321-bdd8-f5f9cc71891a\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") "
Oct 30 01:35:09.810095 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810028    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/71d572c2-09da-4321-bdd8-f5f9cc71891a-kube-api-access-svkn6\" (UniqueName: \"kubernetes.io/projected/71d572c2-09da-4321-bdd8-f5f9cc71891a-kube-api-access-svkn6\") pod \"71d572c2-09da-4321-bdd8-f5f9cc71891a\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") "
Oct 30 01:35:09.810344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810101    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-tmp-dir\") pod \"71d572c2-09da-4321-bdd8-f5f9cc71891a\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") "
Oct 30 01:35:09.810344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810192    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/71d572c2-09da-4321-bdd8-f5f9cc71891a-tls-certs\" (UniqueName: \"kubernetes.io/secret/71d572c2-09da-4321-bdd8-f5f9cc71891a-tls-certs\") pod \"71d572c2-09da-4321-bdd8-f5f9cc71891a\" (UID: \"71d572c2-09da-4321-bdd8-f5f9cc71891a\") "
Oct 30 01:35:09.810344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810219    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-home" pod "71d572c2-09da-4321-bdd8-f5f9cc71891a" (UID: "71d572c2-09da-4321-bdd8-f5f9cc71891a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:35:09.810519 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810402    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-model-cache" pod "71d572c2-09da-4321-bdd8-f5f9cc71891a" (UID: "71d572c2-09da-4321-bdd8-f5f9cc71891a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:35:09.810519 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810415    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-tmp-dir" pod "71d572c2-09da-4321-bdd8-f5f9cc71891a" (UID: "71d572c2-09da-4321-bdd8-f5f9cc71891a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:35:09.810638 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810587    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:09.810638 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810606    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:09.810638 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.810619    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:09.812498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.812473    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-dshm" pod "71d572c2-09da-4321-bdd8-f5f9cc71891a" (UID: "71d572c2-09da-4321-bdd8-f5f9cc71891a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:35:09.813102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.813068    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71d572c2-09da-4321-bdd8-f5f9cc71891a-tls-certs" pod "71d572c2-09da-4321-bdd8-f5f9cc71891a" (UID: "71d572c2-09da-4321-bdd8-f5f9cc71891a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:35:09.813102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.813070    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71d572c2-09da-4321-bdd8-f5f9cc71891a-kube-api-access-svkn6" pod "71d572c2-09da-4321-bdd8-f5f9cc71891a" (UID: "71d572c2-09da-4321-bdd8-f5f9cc71891a"). InnerVolumeSpecName "kube-api-access-svkn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:35:09.911679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.911647    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/71d572c2-09da-4321-bdd8-f5f9cc71891a-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:09.911679 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.911678    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-svkn6\" (UniqueName: \"kubernetes.io/projected/71d572c2-09da-4321-bdd8-f5f9cc71891a-kube-api-access-svkn6\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:09.911887 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:09.911691    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/71d572c2-09da-4321-bdd8-f5f9cc71891a-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:35:10.004296 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.004206    6011 generic.go:356] "Generic (PLEG): container finished" podID="71d572c2-09da-4321-bdd8-f5f9cc71891a" containerID="d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c" exitCode=0
Oct 30 01:35:10.004296 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.004254    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"
Oct 30 01:35:10.004505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.004293    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm" event={"ID":"71d572c2-09da-4321-bdd8-f5f9cc71891a","Type":"ContainerDied","Data":"d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c"}
Oct 30 01:35:10.004505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.004347    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm" event={"ID":"71d572c2-09da-4321-bdd8-f5f9cc71891a","Type":"ContainerDied","Data":"253f37f30e3f4944d5a08e83cd3d4510f55ed924e90b5670439a6b5827a8ff55"}
Oct 30 01:35:10.004505 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.004364    6011 scope.go:122] "RemoveContainer" containerID="d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c"
Oct 30 01:35:10.014234 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.014168    6011 scope.go:122] "RemoveContainer" containerID="d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c"
Oct 30 01:35:10.014531 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:35:10.014504    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c\": container with ID starting with d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c not found: ID does not exist" containerID="d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c"
Oct 30 01:35:10.014622 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.014545    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c"} err="failed to get container status \"d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c\": rpc error: code = NotFound desc = could not find container \"d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c\": container with ID starting with d1e49f7d68666e18adf795f2227818d1830f6274f0c6f01ce0a838779c932a1c not found: ID does not exist"
Oct 30 01:35:10.057787 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.057744    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"]
Oct 30 01:35:10.061884 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:10.061851    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-364d04/simulator-single-main-364d04-kserve-558f95d7f5-qgvwm"]
Oct 30 01:35:12.019260 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:35:12.019216    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="71d572c2-09da-4321-bdd8-f5f9cc71891a" path="/var/lib/kubelet/pods/71d572c2-09da-4321-bdd8-f5f9cc71891a/volumes"
Oct 30 01:35:27.909105 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:35:27.909065    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:35:57.910382 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:35:57.910342    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:36:27.910911 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:36:27.910823    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:36:57.934482 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:36:57.934444    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:37:15.268317 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:15.268269    6011 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-xw7jh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 01:37:15.269081 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:15.269041    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-xw7jh" podUID="a5403c3f-cc0a-434b-9f83-080fe7a76bdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 01:37:27.910543 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:37:27.910501    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:37:32.860412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.860372    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"]
Oct 30 01:37:32.860858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.860812    6011 cpu_manager.go:414] "containerMap: removing container" podUID="71d572c2-09da-4321-bdd8-f5f9cc71891a" containerName="main"
Oct 30 01:37:32.860858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.860824    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="71d572c2-09da-4321-bdd8-f5f9cc71891a" containerName="main"
Oct 30 01:37:32.860858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.860849    6011 cpu_manager.go:414] "containerMap: removing container" podUID="57b51c06-3032-4203-804f-0a8b95bc1daa" containerName="maas-api"
Oct 30 01:37:32.860858 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.860856    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57b51c06-3032-4203-804f-0a8b95bc1daa" containerName="maas-api"
Oct 30 01:37:32.861004 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.860941    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="57b51c06-3032-4203-804f-0a8b95bc1daa" containerName="maas-api"
Oct 30 01:37:32.861004 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.860960    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="71d572c2-09da-4321-bdd8-f5f9cc71891a" containerName="main"
Oct 30 01:37:32.864210 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.864192    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.882921 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.882886    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"]
Oct 30 01:37:32.890143 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.890109    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qphc\" (UniqueName: \"kubernetes.io/projected/acf449b0-4381-4d4b-a154-a973b3dfbaf6-kube-api-access-4qphc\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.890338 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.890161    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.890405 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.890332    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf449b0-4381-4d4b-a154-a973b3dfbaf6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.890405 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.890386    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.893977 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.893950    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-single-main-646191-serving-cert\""
Oct 30 01:37:32.990971 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.990929    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qphc\" (UniqueName: \"kubernetes.io/projected/acf449b0-4381-4d4b-a154-a973b3dfbaf6-kube-api-access-4qphc\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.990971 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.990980    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.991265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.991046    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf449b0-4381-4d4b-a154-a973b3dfbaf6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.991265 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.991239    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.991868 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.991813    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf449b0-4381-4d4b-a154-a973b3dfbaf6-openshift-service-ca\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.993806 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.993784    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-maas-api-tls\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:32.993937 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:32.993913    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-metrics-certs\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:33.012021 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:33.011986    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qphc\" (UniqueName: \"kubernetes.io/projected/acf449b0-4381-4d4b-a154-a973b3dfbaf6-kube-api-access-4qphc\") pod \"maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:33.177803 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:33.177511    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:33.488247 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:33.488210    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"]
Oct 30 01:37:33.490019 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:37:33.489987    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf449b0_4381_4d4b_a154_a973b3dfbaf6.slice/crio-a410f7599021cfbe19c616d25c6f13ea77986fa0ad809f32bf7e3463521be0a9 WatchSource:0}: Error finding container a410f7599021cfbe19c616d25c6f13ea77986fa0ad809f32bf7e3463521be0a9: Status 404 returned error can't find the container with id a410f7599021cfbe19c616d25c6f13ea77986fa0ad809f32bf7e3463521be0a9
Oct 30 01:37:33.818404 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:33.818365    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz" event={"ID":"acf449b0-4381-4d4b-a154-a973b3dfbaf6","Type":"ContainerStarted","Data":"a410f7599021cfbe19c616d25c6f13ea77986fa0ad809f32bf7e3463521be0a9"}
Oct 30 01:37:34.826612 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:34.826572    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz" event={"ID":"acf449b0-4381-4d4b-a154-a973b3dfbaf6","Type":"ContainerStarted","Data":"5bb960bcd107c365872b322a438c97f6ad4e1af7dd1b8c64681506733ff8284e"}
Oct 30 01:37:34.827099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:34.826698    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:34.870057 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:34.870006    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz" podStartSLOduration=2.508115246 podStartE2EDuration="2.869991384s" totalImagesPullingTime="361.876138ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:37:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:37:34.869262205 +0000 UTC m=+3496.972016175" watchObservedRunningTime="2026-10-30 01:37:34.869991384 +0000 UTC m=+3496.972745354"
Oct 30 01:37:40.839021 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:40.838992    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:37:48.037934 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.037897    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"]
Oct 30 01:37:48.041526 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.041504    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.059235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.058902    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"]
Oct 30 01:37:48.069815 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.069767    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-646191\"/\"default-dockercfg-fnfr9\""
Oct 30 01:37:48.071316 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.071286    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-646191\"/\"openshift-service-ca.crt\""
Oct 30 01:37:48.071316 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.071312    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-single-main-646191\"/\"simulator-single-main-646191-kserve-self-signed-certs\""
Oct 30 01:37:48.071560 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.071289    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-single-main-646191\"/\"kube-root-ca.crt\""
Oct 30 01:37:48.146221 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.146164    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-home\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.146412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.146300    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-dshm\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.146412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.146324    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-model-cache\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.146497 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.146406    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-tmp-dir\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.146497 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.146459    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjf88\" (UniqueName: \"kubernetes.io/projected/600f621b-cc0a-45ea-9fac-764336c6cf88-kube-api-access-rjf88\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.146576 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.146517    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/600f621b-cc0a-45ea-9fac-764336c6cf88-tls-certs\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.247473 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.247430    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-home\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.247693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.247502    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-dshm\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.247693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.247530    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-model-cache\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.247693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.247557    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-tmp-dir\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.247693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.247594    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjf88\" (UniqueName: \"kubernetes.io/projected/600f621b-cc0a-45ea-9fac-764336c6cf88-kube-api-access-rjf88\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.247693 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.247625    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/600f621b-cc0a-45ea-9fac-764336c6cf88-tls-certs\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.248020 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.247993    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-tmp-dir\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.248088 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.248007    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-home\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.248088 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.248050    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-model-cache\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.250077 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.250031    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-dshm\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.250220 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.250202    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/600f621b-cc0a-45ea-9fac-764336c6cf88-tls-certs\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.270053 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.270012    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjf88\" (UniqueName: \"kubernetes.io/projected/600f621b-cc0a-45ea-9fac-764336c6cf88-kube-api-access-rjf88\") pod \"simulator-single-main-646191-kserve-7798f44c87-9mgwl\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") " pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.358803 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.358710    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:48.660814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.660781    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"]
Oct 30 01:37:48.661345 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:37:48.661319    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod600f621b_cc0a_45ea_9fac_764336c6cf88.slice/crio-9678028d4f8552599f3cfefd20047c3bd7b1ccfc6727a11567a31fbb74c1e8d3 WatchSource:0}: Error finding container 9678028d4f8552599f3cfefd20047c3bd7b1ccfc6727a11567a31fbb74c1e8d3: Status 404 returned error can't find the container with id 9678028d4f8552599f3cfefd20047c3bd7b1ccfc6727a11567a31fbb74c1e8d3
Oct 30 01:37:48.909994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.909903    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl" event={"ID":"600f621b-cc0a-45ea-9fac-764336c6cf88","Type":"ContainerStarted","Data":"c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181"}
Oct 30 01:37:48.909994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.909950    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl" event={"ID":"600f621b-cc0a-45ea-9fac-764336c6cf88","Type":"ContainerStarted","Data":"9678028d4f8552599f3cfefd20047c3bd7b1ccfc6727a11567a31fbb74c1e8d3"}
Oct 30 01:37:48.953009 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:48.952941    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl" podStartSLOduration=0.9529252 podStartE2EDuration="952.9252ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:37:48.950533663 +0000 UTC m=+3511.053287634" watchObservedRunningTime="2026-10-30 01:37:48.9529252 +0000 UTC m=+3511.055679272"
Oct 30 01:37:49.358918 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:49.358876    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:57.909454 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:37:57.909412    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:37:58.359085 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:58.359044    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:58.372023 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:58.371992    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:37:58.975431 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:37:58.975402    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:38:23.850138 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:23.850097    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"]
Oct 30 01:38:27.910620 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:38:27.910581    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:38:29.803715 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:29.803668    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"]
Oct 30 01:38:29.804366 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:29.803956    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz" podUID="acf449b0-4381-4d4b-a154-a973b3dfbaf6" containerName="maas-api" containerID="cri-o://5bb960bcd107c365872b322a438c97f6ad4e1af7dd1b8c64681506733ff8284e" gracePeriod=30
Oct 30 01:38:30.134239 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.134202    6011 generic.go:356] "Generic (PLEG): container finished" podID="acf449b0-4381-4d4b-a154-a973b3dfbaf6" containerID="5bb960bcd107c365872b322a438c97f6ad4e1af7dd1b8c64681506733ff8284e" exitCode=0
Oct 30 01:38:30.134426 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.134271    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz" event={"ID":"acf449b0-4381-4d4b-a154-a973b3dfbaf6","Type":"ContainerDied","Data":"5bb960bcd107c365872b322a438c97f6ad4e1af7dd1b8c64681506733ff8284e"}
Oct 30 01:38:30.280474 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.280442    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:38:30.380005 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.379900    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-metrics-certs\") pod \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") "
Oct 30 01:38:30.380005 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.379968    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/acf449b0-4381-4d4b-a154-a973b3dfbaf6-kube-api-access-4qphc\" (UniqueName: \"kubernetes.io/projected/acf449b0-4381-4d4b-a154-a973b3dfbaf6-kube-api-access-4qphc\") pod \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") "
Oct 30 01:38:30.380280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.380030    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/acf449b0-4381-4d4b-a154-a973b3dfbaf6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf449b0-4381-4d4b-a154-a973b3dfbaf6-openshift-service-ca\") pod \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") "
Oct 30 01:38:30.380280 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.380096    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-maas-api-tls\") pod \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\" (UID: \"acf449b0-4381-4d4b-a154-a973b3dfbaf6\") "
Oct 30 01:38:30.381312 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.381262    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/acf449b0-4381-4d4b-a154-a973b3dfbaf6-openshift-service-ca" pod "acf449b0-4381-4d4b-a154-a973b3dfbaf6" (UID: "acf449b0-4381-4d4b-a154-a973b3dfbaf6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:38:30.382573 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.382536    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-metrics-certs" pod "acf449b0-4381-4d4b-a154-a973b3dfbaf6" (UID: "acf449b0-4381-4d4b-a154-a973b3dfbaf6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:38:30.382719 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.382701    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-maas-api-tls" pod "acf449b0-4381-4d4b-a154-a973b3dfbaf6" (UID: "acf449b0-4381-4d4b-a154-a973b3dfbaf6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:38:30.383030 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.383002    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/acf449b0-4381-4d4b-a154-a973b3dfbaf6-kube-api-access-4qphc" pod "acf449b0-4381-4d4b-a154-a973b3dfbaf6" (UID: "acf449b0-4381-4d4b-a154-a973b3dfbaf6"). InnerVolumeSpecName "kube-api-access-4qphc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:38:30.481802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.481756    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:30.481802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.481796    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/acf449b0-4381-4d4b-a154-a973b3dfbaf6-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:30.481802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.481807    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qphc\" (UniqueName: \"kubernetes.io/projected/acf449b0-4381-4d4b-a154-a973b3dfbaf6-kube-api-access-4qphc\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:30.482068 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:30.481817    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/acf449b0-4381-4d4b-a154-a973b3dfbaf6-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:31.142201 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:31.142149    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"
Oct 30 01:38:31.142645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:31.142149    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz" event={"ID":"acf449b0-4381-4d4b-a154-a973b3dfbaf6","Type":"ContainerDied","Data":"a410f7599021cfbe19c616d25c6f13ea77986fa0ad809f32bf7e3463521be0a9"}
Oct 30 01:38:31.142645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:31.142308    6011 scope.go:122] "RemoveContainer" containerID="5bb960bcd107c365872b322a438c97f6ad4e1af7dd1b8c64681506733ff8284e"
Oct 30 01:38:31.196364 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:31.196328    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"]
Oct 30 01:38:31.209849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:31.209813    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-single-main-646191-5695fbfcb4-x2zbz"]
Oct 30 01:38:32.020677 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:32.020642    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="acf449b0-4381-4d4b-a154-a973b3dfbaf6" path="/var/lib/kubelet/pods/acf449b0-4381-4d4b-a154-a973b3dfbaf6/volumes"
Oct 30 01:38:38.875570 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:38.872002    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl" podUID="600f621b-cc0a-45ea-9fac-764336c6cf88" containerName="main" containerID="cri-o://c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181" gracePeriod=45
Oct 30 01:38:38.964741 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:38.964702    6011 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl" podUID="600f621b-cc0a-45ea-9fac-764336c6cf88" containerName="main" probeResult="failure" output="Get \"https://10.134.0.162:8000/ready\": dial tcp 10.134.0.162:8000: connect: connection refused"
Oct 30 01:38:39.130304 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.130248    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:38:39.193717 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.193676    6011 generic.go:356] "Generic (PLEG): container finished" podID="600f621b-cc0a-45ea-9fac-764336c6cf88" containerID="c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181" exitCode=0
Oct 30 01:38:39.193717 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.193722    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"
Oct 30 01:38:39.193994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.193773    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl" event={"ID":"600f621b-cc0a-45ea-9fac-764336c6cf88","Type":"ContainerDied","Data":"c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181"}
Oct 30 01:38:39.193994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.193817    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl" event={"ID":"600f621b-cc0a-45ea-9fac-764336c6cf88","Type":"ContainerDied","Data":"9678028d4f8552599f3cfefd20047c3bd7b1ccfc6727a11567a31fbb74c1e8d3"}
Oct 30 01:38:39.193994 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.193840    6011 scope.go:122] "RemoveContainer" containerID="c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181"
Oct 30 01:38:39.205235 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.205210    6011 scope.go:122] "RemoveContainer" containerID="c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181"
Oct 30 01:38:39.205522 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:38:39.205501    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181\": container with ID starting with c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181 not found: ID does not exist" containerID="c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181"
Oct 30 01:38:39.205619 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.205538    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181"} err="failed to get container status \"c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181\": rpc error: code = NotFound desc = could not find container \"c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181\": container with ID starting with c33484a25edfb840a4cc6bd6faf3a067d76aaacdfcbafa5d113e536f15d5f181 not found: ID does not exist"
Oct 30 01:38:39.270735 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.270696    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/600f621b-cc0a-45ea-9fac-764336c6cf88-tls-certs\" (UniqueName: \"kubernetes.io/secret/600f621b-cc0a-45ea-9fac-764336c6cf88-tls-certs\") pod \"600f621b-cc0a-45ea-9fac-764336c6cf88\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") "
Oct 30 01:38:39.270910 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.270760    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-home\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-home\") pod \"600f621b-cc0a-45ea-9fac-764336c6cf88\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") "
Oct 30 01:38:39.270910 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.270897    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-model-cache\") pod \"600f621b-cc0a-45ea-9fac-764336c6cf88\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") "
Oct 30 01:38:39.270991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.270931    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-dshm\") pod \"600f621b-cc0a-45ea-9fac-764336c6cf88\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") "
Oct 30 01:38:39.270991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.270957    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/600f621b-cc0a-45ea-9fac-764336c6cf88-kube-api-access-rjf88\" (UniqueName: \"kubernetes.io/projected/600f621b-cc0a-45ea-9fac-764336c6cf88-kube-api-access-rjf88\") pod \"600f621b-cc0a-45ea-9fac-764336c6cf88\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") "
Oct 30 01:38:39.270991 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.270979    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-tmp-dir\") pod \"600f621b-cc0a-45ea-9fac-764336c6cf88\" (UID: \"600f621b-cc0a-45ea-9fac-764336c6cf88\") "
Oct 30 01:38:39.271158 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.271022    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-home" pod "600f621b-cc0a-45ea-9fac-764336c6cf88" (UID: "600f621b-cc0a-45ea-9fac-764336c6cf88"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:38:39.271266 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.271241    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-model-cache" pod "600f621b-cc0a-45ea-9fac-764336c6cf88" (UID: "600f621b-cc0a-45ea-9fac-764336c6cf88"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:38:39.271377 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.271356    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:39.271419 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.271383    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:39.271683 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.271655    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-tmp-dir" pod "600f621b-cc0a-45ea-9fac-764336c6cf88" (UID: "600f621b-cc0a-45ea-9fac-764336c6cf88"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:38:39.273214 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.273193    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-dshm" pod "600f621b-cc0a-45ea-9fac-764336c6cf88" (UID: "600f621b-cc0a-45ea-9fac-764336c6cf88"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:38:39.273355 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.273338    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/600f621b-cc0a-45ea-9fac-764336c6cf88-kube-api-access-rjf88" pod "600f621b-cc0a-45ea-9fac-764336c6cf88" (UID: "600f621b-cc0a-45ea-9fac-764336c6cf88"). InnerVolumeSpecName "kube-api-access-rjf88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:38:39.273411 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.273349    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/600f621b-cc0a-45ea-9fac-764336c6cf88-tls-certs" pod "600f621b-cc0a-45ea-9fac-764336c6cf88" (UID: "600f621b-cc0a-45ea-9fac-764336c6cf88"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:38:39.372296 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.372263    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/600f621b-cc0a-45ea-9fac-764336c6cf88-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:39.372296 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.372292    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:39.372296 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.372302    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjf88\" (UniqueName: \"kubernetes.io/projected/600f621b-cc0a-45ea-9fac-764336c6cf88-kube-api-access-rjf88\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:39.372563 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.372314    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/600f621b-cc0a-45ea-9fac-764336c6cf88-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:38:39.541294 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.541256    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"]
Oct 30 01:38:39.544384 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:39.544356    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-single-main-646191/simulator-single-main-646191-kserve-7798f44c87-9mgwl"]
Oct 30 01:38:40.029084 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:40.029042    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="600f621b-cc0a-45ea-9fac-764336c6cf88" path="/var/lib/kubelet/pods/600f621b-cc0a-45ea-9fac-764336c6cf88/volumes"
Oct 30 01:38:42.233339 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.233303    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"]
Oct 30 01:38:42.233999 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.233978    6011 cpu_manager.go:414] "containerMap: removing container" podUID="acf449b0-4381-4d4b-a154-a973b3dfbaf6" containerName="maas-api"
Oct 30 01:38:42.234094 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.234001    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="acf449b0-4381-4d4b-a154-a973b3dfbaf6" containerName="maas-api"
Oct 30 01:38:42.234094 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.234017    6011 cpu_manager.go:414] "containerMap: removing container" podUID="600f621b-cc0a-45ea-9fac-764336c6cf88" containerName="main"
Oct 30 01:38:42.234094 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.234024    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="600f621b-cc0a-45ea-9fac-764336c6cf88" containerName="main"
Oct 30 01:38:42.234281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.234151    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="acf449b0-4381-4d4b-a154-a973b3dfbaf6" containerName="maas-api"
Oct 30 01:38:42.234281 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.234165    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="600f621b-cc0a-45ea-9fac-764336c6cf88" containerName="main"
Oct 30 01:38:42.238737 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.238713    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.253756 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.253724    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"]
Oct 30 01:38:42.265902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.265863    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-fa62f153-kserve-self-signed-certs\""
Oct 30 01:38:42.405753 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.405692    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-home\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.405968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.405804    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w72db\" (UniqueName: \"kubernetes.io/projected/63c72e60-8425-4d41-825b-e53ff16da154-kube-api-access-w72db\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.405968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.405890    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-tmp-dir\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.405968 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.405927    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-model-cache\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.406330 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.406109    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-dshm\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.406330 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.406148    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c72e60-8425-4d41-825b-e53ff16da154-tls-certs\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.507572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.507476    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-dshm\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.507572 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.507523    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c72e60-8425-4d41-825b-e53ff16da154-tls-certs\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.507782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.507639    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-home\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.507782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.507698    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w72db\" (UniqueName: \"kubernetes.io/projected/63c72e60-8425-4d41-825b-e53ff16da154-kube-api-access-w72db\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.507782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.507750    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-tmp-dir\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.507782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.507774    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-model-cache\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.508041 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.508008    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-home\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.508220 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.508100    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-tmp-dir\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.508293 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.508210    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-model-cache\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.510056 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.510035    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-dshm\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.510358 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.510338    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c72e60-8425-4d41-825b-e53ff16da154-tls-certs\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.528766 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.528736    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w72db\" (UniqueName: \"kubernetes.io/projected/63c72e60-8425-4d41-825b-e53ff16da154-kube-api-access-w72db\") pod \"e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") " pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.554687 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.554645    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:42.855465 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:42.855433    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"]
Oct 30 01:38:42.855589 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:38:42.855563    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63c72e60_8425_4d41_825b_e53ff16da154.slice/crio-c9ca1cfd0e2661d3dd9c4c78f203def2ffe092033b4a630209868a0d5cbf54dc WatchSource:0}: Error finding container c9ca1cfd0e2661d3dd9c4c78f203def2ffe092033b4a630209868a0d5cbf54dc: Status 404 returned error can't find the container with id c9ca1cfd0e2661d3dd9c4c78f203def2ffe092033b4a630209868a0d5cbf54dc
Oct 30 01:38:43.219403 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:43.219313    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75" event={"ID":"63c72e60-8425-4d41-825b-e53ff16da154","Type":"ContainerStarted","Data":"367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56"}
Oct 30 01:38:43.219403 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:43.219365    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75" event={"ID":"63c72e60-8425-4d41-825b-e53ff16da154","Type":"ContainerStarted","Data":"c9ca1cfd0e2661d3dd9c4c78f203def2ffe092033b4a630209868a0d5cbf54dc"}
Oct 30 01:38:43.260908 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:43.260842    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75" podStartSLOduration=1.260829684 podStartE2EDuration="1.260829684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:38:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:38:43.259706043 +0000 UTC m=+3565.362460013" watchObservedRunningTime="2026-10-30 01:38:43.260829684 +0000 UTC m=+3565.363583654"
Oct 30 01:38:43.555760 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:43.555725    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:52.555091 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:52.555041    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:52.568247 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:52.568210    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:53.293406 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:38:53.293377    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:38:57.910117 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:38:57.910078    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:39:04.426709 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.426678    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"]
Oct 30 01:39:04.430148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.430130    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.448203 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.448133    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"]
Oct 30 01:39:04.458498 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.458463    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-fa62f153-kserve-self-signed-certs\""
Oct 30 01:39:04.495344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.495312    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-dshm\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.495532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.495364    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-home\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.495532 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.495435    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fckgj\" (UniqueName: \"kubernetes.io/projected/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-kube-api-access-fckgj\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.495635 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.495547    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tls-certs\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.495635 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.495593    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-model-cache\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.495635 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.495612    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tmp-dir\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.596586 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.596548    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-dshm\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.596752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.596600    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-home\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.596752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.596643    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fckgj\" (UniqueName: \"kubernetes.io/projected/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-kube-api-access-fckgj\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.596752 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.596721    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tls-certs\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.596919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.596764    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-model-cache\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.596919 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.596788    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tmp-dir\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.597124 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.597099    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-home\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.597226 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.597124    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-model-cache\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.597273 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.597219    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tmp-dir\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.599007 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.598985    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-dshm\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.599416 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.599391    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tls-certs\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.618055 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.618024    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fckgj\" (UniqueName: \"kubernetes.io/projected/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-kube-api-access-fckgj\") pod \"e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") " pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:04.745782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:04.745691    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:05.043034 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:05.043007    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"]
Oct 30 01:39:05.043719 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:39:05.043692    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1ef59ab9_5a51_4f95_aa9b_c102e2905dbe.slice/crio-a1c848d8017feb49a368f6039cf542b615dde15f951bb1209d9db1b09d593d99 WatchSource:0}: Error finding container a1c848d8017feb49a368f6039cf542b615dde15f951bb1209d9db1b09d593d99: Status 404 returned error can't find the container with id a1c848d8017feb49a368f6039cf542b615dde15f951bb1209d9db1b09d593d99
Oct 30 01:39:05.352139 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:05.352053    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228" event={"ID":"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe","Type":"ContainerStarted","Data":"c702718c380e17234fa3a7a0bfabb5853636042dabb7a47e32e65a01e23ff7b6"}
Oct 30 01:39:05.352139 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:05.352098    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228" event={"ID":"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe","Type":"ContainerStarted","Data":"a1c848d8017feb49a368f6039cf542b615dde15f951bb1209d9db1b09d593d99"}
Oct 30 01:39:05.393989 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:05.393936    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228" podStartSLOduration=1.393907176 podStartE2EDuration="1.393907176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:39:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:39:05.393419518 +0000 UTC m=+3587.496173488" watchObservedRunningTime="2026-10-30 01:39:05.393907176 +0000 UTC m=+3587.496661147"
Oct 30 01:39:05.746250 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:05.746134    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:14.746889 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:14.746846    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:14.759668 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:14.759642    6011 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:15.459781 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:15.459747    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:27.909588 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:39:27.909506    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:39:31.811168 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:31.811123    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"]
Oct 30 01:39:34.561630 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:34.561589    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"]
Oct 30 01:39:46.832367 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:46.832301    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75" podUID="63c72e60-8425-4d41-825b-e53ff16da154" containerName="main" containerID="cri-o://367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56" gracePeriod=45
Oct 30 01:39:47.094207 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.094112    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:39:47.200341 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200299    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-model-cache\") pod \"63c72e60-8425-4d41-825b-e53ff16da154\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") "
Oct 30 01:39:47.200564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200392    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-dshm\") pod \"63c72e60-8425-4d41-825b-e53ff16da154\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") "
Oct 30 01:39:47.200564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200437    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-home\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-home\") pod \"63c72e60-8425-4d41-825b-e53ff16da154\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") "
Oct 30 01:39:47.200564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200504    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-tmp-dir\") pod \"63c72e60-8425-4d41-825b-e53ff16da154\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") "
Oct 30 01:39:47.200564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200534    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63c72e60-8425-4d41-825b-e53ff16da154-kube-api-access-w72db\" (UniqueName: \"kubernetes.io/projected/63c72e60-8425-4d41-825b-e53ff16da154-kube-api-access-w72db\") pod \"63c72e60-8425-4d41-825b-e53ff16da154\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") "
Oct 30 01:39:47.200802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200586    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63c72e60-8425-4d41-825b-e53ff16da154-tls-certs\" (UniqueName: \"kubernetes.io/secret/63c72e60-8425-4d41-825b-e53ff16da154-tls-certs\") pod \"63c72e60-8425-4d41-825b-e53ff16da154\" (UID: \"63c72e60-8425-4d41-825b-e53ff16da154\") "
Oct 30 01:39:47.200802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200631    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-model-cache" pod "63c72e60-8425-4d41-825b-e53ff16da154" (UID: "63c72e60-8425-4d41-825b-e53ff16da154"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:47.200802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200732    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-tmp-dir" pod "63c72e60-8425-4d41-825b-e53ff16da154" (UID: "63c72e60-8425-4d41-825b-e53ff16da154"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:47.200802 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200776    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-home" pod "63c72e60-8425-4d41-825b-e53ff16da154" (UID: "63c72e60-8425-4d41-825b-e53ff16da154"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:47.200996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200938    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:47.200996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200956    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:47.200996 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.200965    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:47.202956 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.202924    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c72e60-8425-4d41-825b-e53ff16da154-kube-api-access-w72db" pod "63c72e60-8425-4d41-825b-e53ff16da154" (UID: "63c72e60-8425-4d41-825b-e53ff16da154"). InnerVolumeSpecName "kube-api-access-w72db". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:39:47.203076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.202974    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-dshm" pod "63c72e60-8425-4d41-825b-e53ff16da154" (UID: "63c72e60-8425-4d41-825b-e53ff16da154"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:47.203076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.202983    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63c72e60-8425-4d41-825b-e53ff16da154-tls-certs" pod "63c72e60-8425-4d41-825b-e53ff16da154" (UID: "63c72e60-8425-4d41-825b-e53ff16da154"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:39:47.301669 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.301632    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w72db\" (UniqueName: \"kubernetes.io/projected/63c72e60-8425-4d41-825b-e53ff16da154-kube-api-access-w72db\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:47.301669 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.301667    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c72e60-8425-4d41-825b-e53ff16da154-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:47.301669 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.301677    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c72e60-8425-4d41-825b-e53ff16da154-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:47.702118 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.702083    6011 generic.go:356] "Generic (PLEG): container finished" podID="63c72e60-8425-4d41-825b-e53ff16da154" containerID="367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56" exitCode=0
Oct 30 01:39:47.702333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.702141    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"
Oct 30 01:39:47.702333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.702186    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75" event={"ID":"63c72e60-8425-4d41-825b-e53ff16da154","Type":"ContainerDied","Data":"367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56"}
Oct 30 01:39:47.702333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.702235    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75" event={"ID":"63c72e60-8425-4d41-825b-e53ff16da154","Type":"ContainerDied","Data":"c9ca1cfd0e2661d3dd9c4c78f203def2ffe092033b4a630209868a0d5cbf54dc"}
Oct 30 01:39:47.702333 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.702253    6011 scope.go:122] "RemoveContainer" containerID="367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56"
Oct 30 01:39:47.712760 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.712741    6011 scope.go:122] "RemoveContainer" containerID="367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56"
Oct 30 01:39:47.713017 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:39:47.712997    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56\": container with ID starting with 367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56 not found: ID does not exist" containerID="367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56"
Oct 30 01:39:47.713068 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.713028    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56"} err="failed to get container status \"367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56\": rpc error: code = NotFound desc = could not find container \"367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56\": container with ID starting with 367c3d34ca7510b2923cc454e1fc0c49727d2c2c1a1200c067fd208a93ad7c56 not found: ID does not exist"
Oct 30 01:39:47.756794 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.756752    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"]
Oct 30 01:39:47.770399 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:47.770359    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-fa62f153-kserve-7954664b69-jqh75"]
Oct 30 01:39:48.020215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:48.020124    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63c72e60-8425-4d41-825b-e53ff16da154" path="/var/lib/kubelet/pods/63c72e60-8425-4d41-825b-e53ff16da154/volumes"
Oct 30 01:39:49.593856 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:49.593815    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228" podUID="1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" containerName="main" containerID="cri-o://c702718c380e17234fa3a7a0bfabb5853636042dabb7a47e32e65a01e23ff7b6" gracePeriod=45
Oct 30 01:39:49.723798 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:49.723747    6011 generic.go:356] "Generic (PLEG): container finished" podID="1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" containerID="c702718c380e17234fa3a7a0bfabb5853636042dabb7a47e32e65a01e23ff7b6" exitCode=0
Oct 30 01:39:49.723960 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:49.723829    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228" event={"ID":"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe","Type":"ContainerDied","Data":"c702718c380e17234fa3a7a0bfabb5853636042dabb7a47e32e65a01e23ff7b6"}
Oct 30 01:39:49.933616 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:49.933575    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:50.030564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.030525    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-dshm\") pod \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") "
Oct 30 01:39:50.030768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.030585    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tmp-dir\") pod \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") "
Oct 30 01:39:50.030768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.030651    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tls-certs\") pod \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") "
Oct 30 01:39:50.030768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.030685    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-home\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-home\") pod \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") "
Oct 30 01:39:50.030768 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.030721    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-model-cache\") pod \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") "
Oct 30 01:39:50.031099 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.030780    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-kube-api-access-fckgj\" (UniqueName: \"kubernetes.io/projected/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-kube-api-access-fckgj\") pod \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\" (UID: \"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe\") "
Oct 30 01:39:50.031769 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.031735    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-home" pod "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" (UID: "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:50.031960 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.031938    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-model-cache" pod "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" (UID: "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:50.032149 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.032128    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tmp-dir" pod "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" (UID: "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:50.034511 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.034484    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-kube-api-access-fckgj" pod "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" (UID: "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe"). InnerVolumeSpecName "kube-api-access-fckgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:39:50.034900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.034797    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tls-certs" pod "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" (UID: "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:39:50.034900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.034869    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-dshm" pod "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" (UID: "1ef59ab9-5a51-4f95-aa9b-c102e2905dbe"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 01:39:50.131932 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.131839    6011 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-dshm\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:50.131932 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.131871    6011 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tmp-dir\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:50.131932 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.131883    6011 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-tls-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:50.131932 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.131893    6011 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-home\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:50.131932 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.131904    6011 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-model-cache\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:50.131932 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.131914    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fckgj\" (UniqueName: \"kubernetes.io/projected/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe-kube-api-access-fckgj\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:39:50.731032 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.731001    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"
Oct 30 01:39:50.731524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.731001    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228" event={"ID":"1ef59ab9-5a51-4f95-aa9b-c102e2905dbe","Type":"ContainerDied","Data":"a1c848d8017feb49a368f6039cf542b615dde15f951bb1209d9db1b09d593d99"}
Oct 30 01:39:50.731524 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.731116    6011 scope.go:122] "RemoveContainer" containerID="c702718c380e17234fa3a7a0bfabb5853636042dabb7a47e32e65a01e23ff7b6"
Oct 30 01:39:50.809205 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.802729    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"]
Oct 30 01:39:50.816501 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:50.816465    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-fa62f153-kserve-6ddbc77fd9-2x228"]
Oct 30 01:39:52.019965 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:39:52.019926    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" path="/var/lib/kubelet/pods/1ef59ab9-5a51-4f95-aa9b-c102e2905dbe/volumes"
Oct 30 01:39:57.910465 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:39:57.910426    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:40:13.918252 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.918218    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-xsksf"]
Oct 30 01:40:13.918730 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.918698    6011 cpu_manager.go:414] "containerMap: removing container" podUID="1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" containerName="main"
Oct 30 01:40:13.918730 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.918717    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" containerName="main"
Oct 30 01:40:13.918866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.918745    6011 cpu_manager.go:414] "containerMap: removing container" podUID="63c72e60-8425-4d41-825b-e53ff16da154" containerName="main"
Oct 30 01:40:13.918866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.918752    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63c72e60-8425-4d41-825b-e53ff16da154" containerName="main"
Oct 30 01:40:13.918866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.918838    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="1ef59ab9-5a51-4f95-aa9b-c102e2905dbe" containerName="main"
Oct 30 01:40:13.918866 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.918855    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="63c72e60-8425-4d41-825b-e53ff16da154" containerName="main"
Oct 30 01:40:13.923445 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.923419    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:13.941468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.941438    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-xsksf"]
Oct 30 01:40:13.985380 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.985343    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4zwq\" (UniqueName: \"kubernetes.io/projected/e9840e72-b47a-4fb6-b260-515f0f35bfcb-kube-api-access-g4zwq\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:13.985541 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.985470    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9840e72-b47a-4fb6-b260-515f0f35bfcb-openshift-service-ca\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:13.985541 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.985494    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9840e72-b47a-4fb6-b260-515f0f35bfcb-maas-api-tls\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:13.985615 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:13.985588    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9840e72-b47a-4fb6-b260-515f0f35bfcb-metrics-certs\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.086347 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.086309    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9840e72-b47a-4fb6-b260-515f0f35bfcb-openshift-service-ca\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.086606 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.086359    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9840e72-b47a-4fb6-b260-515f0f35bfcb-maas-api-tls\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.086606 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.086453    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9840e72-b47a-4fb6-b260-515f0f35bfcb-metrics-certs\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.086606 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.086507    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4zwq\" (UniqueName: \"kubernetes.io/projected/e9840e72-b47a-4fb6-b260-515f0f35bfcb-kube-api-access-g4zwq\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.087283 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.087255    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e9840e72-b47a-4fb6-b260-515f0f35bfcb-openshift-service-ca\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.089155 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.089128    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9840e72-b47a-4fb6-b260-515f0f35bfcb-metrics-certs\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.089296 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.089207    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e9840e72-b47a-4fb6-b260-515f0f35bfcb-maas-api-tls\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.107394 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.107368    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4zwq\" (UniqueName: \"kubernetes.io/projected/e9840e72-b47a-4fb6-b260-515f0f35bfcb-kube-api-access-g4zwq\") pod \"maas-api-595cc648c-xsksf\" (UID: \"e9840e72-b47a-4fb6-b260-515f0f35bfcb\") " pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.240665 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.240594    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:14.538727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.538696    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-595cc648c-xsksf"]
Oct 30 01:40:14.539273 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:40:14.539246    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9840e72_b47a_4fb6_b260_515f0f35bfcb.slice/crio-1ae6eed95d933e880aafac1d6ff27c3619e3315c443ecc3e2d7ef2a0cd3d7f7f WatchSource:0}: Error finding container 1ae6eed95d933e880aafac1d6ff27c3619e3315c443ecc3e2d7ef2a0cd3d7f7f: Status 404 returned error can't find the container with id 1ae6eed95d933e880aafac1d6ff27c3619e3315c443ecc3e2d7ef2a0cd3d7f7f
Oct 30 01:40:14.540529 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.540512    6011 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 01:40:14.898782 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:14.898706    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf" event={"ID":"e9840e72-b47a-4fb6-b260-515f0f35bfcb","Type":"ContainerStarted","Data":"1ae6eed95d933e880aafac1d6ff27c3619e3315c443ecc3e2d7ef2a0cd3d7f7f"}
Oct 30 01:40:15.906109 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:15.906067    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf" event={"ID":"e9840e72-b47a-4fb6-b260-515f0f35bfcb","Type":"ContainerStarted","Data":"8835b2982c41a76bd7af9067905db8e598b88234458ac8a4c10a71d8788565dc"}
Oct 30 01:40:15.906582 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:15.906203    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:15.950610 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:15.950557    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf" podStartSLOduration=2.478688938 podStartE2EDuration="2.950543264s" totalImagesPullingTime="471.854326ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:40:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:40:15.94968281 +0000 UTC m=+3658.052436780" watchObservedRunningTime="2026-10-30 01:40:15.950543264 +0000 UTC m=+3658.053297235"
Oct 30 01:40:21.916906 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:21.916873    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-595cc648c-xsksf"
Oct 30 01:40:21.989828 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:21.989784    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"]
Oct 30 01:40:21.990113 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:21.990059    6011 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd" podUID="70d5a52c-2627-4e9c-9229-01a752fc0827" containerName="maas-api" containerID="cri-o://5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df" gracePeriod=30
Oct 30 01:40:23.274601 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.274573    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 01:40:23.375240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.375099    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70d5a52c-2627-4e9c-9229-01a752fc0827-kube-api-access-92549\" (UniqueName: \"kubernetes.io/projected/70d5a52c-2627-4e9c-9229-01a752fc0827-kube-api-access-92549\") pod \"70d5a52c-2627-4e9c-9229-01a752fc0827\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") "
Oct 30 01:40:23.375240 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.375215    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70d5a52c-2627-4e9c-9229-01a752fc0827-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70d5a52c-2627-4e9c-9229-01a752fc0827-openshift-service-ca\") pod \"70d5a52c-2627-4e9c-9229-01a752fc0827\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") "
Oct 30 01:40:23.375485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.375288    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-metrics-certs\") pod \"70d5a52c-2627-4e9c-9229-01a752fc0827\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") "
Oct 30 01:40:23.375485 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.375450    6011 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-maas-api-tls\") pod \"70d5a52c-2627-4e9c-9229-01a752fc0827\" (UID: \"70d5a52c-2627-4e9c-9229-01a752fc0827\") "
Oct 30 01:40:23.375922 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.375870    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70d5a52c-2627-4e9c-9229-01a752fc0827-openshift-service-ca" pod "70d5a52c-2627-4e9c-9229-01a752fc0827" (UID: "70d5a52c-2627-4e9c-9229-01a752fc0827"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 01:40:23.377800 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.377771    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-metrics-certs" pod "70d5a52c-2627-4e9c-9229-01a752fc0827" (UID: "70d5a52c-2627-4e9c-9229-01a752fc0827"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:40:23.378129 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.378096    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70d5a52c-2627-4e9c-9229-01a752fc0827-kube-api-access-92549" pod "70d5a52c-2627-4e9c-9229-01a752fc0827" (UID: "70d5a52c-2627-4e9c-9229-01a752fc0827"). InnerVolumeSpecName "kube-api-access-92549". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 01:40:23.378311 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.378282    6011 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-maas-api-tls" pod "70d5a52c-2627-4e9c-9229-01a752fc0827" (UID: "70d5a52c-2627-4e9c-9229-01a752fc0827"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 01:40:23.477123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.477077    6011 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-maas-api-tls\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:40:23.477123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.477110    6011 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92549\" (UniqueName: \"kubernetes.io/projected/70d5a52c-2627-4e9c-9229-01a752fc0827-kube-api-access-92549\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:40:23.477123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.477123    6011 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70d5a52c-2627-4e9c-9229-01a752fc0827-openshift-service-ca\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:40:23.477123 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.477133    6011 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70d5a52c-2627-4e9c-9229-01a752fc0827-metrics-certs\") on node \"ip-10-0-131-254.ec2.internal\" DevicePath \"\""
Oct 30 01:40:23.993202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.992747    6011 generic.go:356] "Generic (PLEG): container finished" podID="70d5a52c-2627-4e9c-9229-01a752fc0827" containerID="5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df" exitCode=0
Oct 30 01:40:23.993202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.992863    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd" event={"ID":"70d5a52c-2627-4e9c-9229-01a752fc0827","Type":"ContainerDied","Data":"5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df"}
Oct 30 01:40:23.993202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.992917    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd" event={"ID":"70d5a52c-2627-4e9c-9229-01a752fc0827","Type":"ContainerDied","Data":"5b443c5baed29248244dc9d47594384fc5a11a2c0023f650c0f45b5c4bb6d2bb"}
Oct 30 01:40:23.993202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.992947    6011 scope.go:122] "RemoveContainer" containerID="5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df"
Oct 30 01:40:23.993202 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:23.993099    6011 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"
Oct 30 01:40:24.028489 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:24.028462    6011 scope.go:122] "RemoveContainer" containerID="5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df"
Oct 30 01:40:24.028827 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:40:24.028804    6011 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df\": container with ID starting with 5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df not found: ID does not exist" containerID="5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df"
Oct 30 01:40:24.028895 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:24.028837    6011 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df"} err="failed to get container status \"5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df\": rpc error: code = NotFound desc = could not find container \"5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df\": container with ID starting with 5a90fdd9eab7a26e7a759a591dd4051bb6b57afe74678b6dfcd4bfa378f630df not found: ID does not exist"
Oct 30 01:40:24.066215 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:24.064800    6011 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"]
Oct 30 01:40:24.081214 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:24.080977    6011 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-6h4cd"]
Oct 30 01:40:26.021609 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:40:26.021574    6011 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70d5a52c-2627-4e9c-9229-01a752fc0827" path="/var/lib/kubelet/pods/70d5a52c-2627-4e9c-9229-01a752fc0827/volumes"
Oct 30 01:40:27.910439 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:40:27.910398    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:40:57.910447 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:40:57.910403    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:41:11.970625 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:11.970584    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-794dc9cdb5-f6dgm_d2949671-a478-4bd4-9da8-d02e13ad22f2/authorino/0.log"
Oct 30 01:41:24.068658 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:24.068631    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8cc7fc69d-vbxkf_aed71d7a-a307-474d-8532-87e110b81265/copy-manifests/0.log"
Oct 30 01:41:24.098589 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:24.098536    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8cc7fc69d-vbxkf_aed71d7a-a307-474d-8532-87e110b81265/manager/0.log"
Oct 30 01:41:25.458112 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:25.458080    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-xsksf_e9840e72-b47a-4fb6-b260-515f0f35bfcb/maas-api/0.log"
Oct 30 01:41:27.262642 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:27.262590    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-t92zl_f6cc5223-d801-42fd-82d0-f854c7a44be4/postgres/0.log"
Oct 30 01:41:27.910767 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:41:27.910730    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:41:28.669519 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:28.669471    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-794dc9cdb5-f6dgm_d2949671-a478-4bd4-9da8-d02e13ad22f2/authorino/0.log"
Oct 30 01:41:29.116610 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:29.116582    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-lsd2f_869d4394-feed-4576-a544-4dc0b54c1018/manager/0.log"
Oct 30 01:41:29.292444 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:29.292415    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-ltv2b_93246f44-8c47-4f83-a435-0ec11e3d4436/kuadrant-console-plugin/0.log"
Oct 30 01:41:31.228042 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:31.228004    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7597b65cbf-stpb4_dcad1ee6-9300-4dd7-98fd-6a05d3a6b867/kube-auth-proxy/0.log"
Oct 30 01:41:31.785873 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:31.785823    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-4dwdf_f3f3f72a-7fe6-4495-a7e8-61f971757191/istio-proxy/0.log"
Oct 30 01:41:32.550899 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:32.550856    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-nnnk8_a730d256-143d-4ee5-91ed-e099a288a7f5/istio-proxy/0.log"
Oct 30 01:41:33.131499 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:33.131453    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-xw7jh_a5403c3f-cc0a-434b-9f83-080fe7a76bdb/istio-proxy/0.log"
Oct 30 01:41:34.790425 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:34.790375    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-5bhqz_c65bb226-5405-477b-9730-3d33e151004c/main/0.log"
Oct 30 01:41:34.962356 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:34.962326    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-sfwrr_2d2fef78-7a92-4353-8c09-33b082f9d7b8/main/0.log"
Oct 30 01:41:35.516690 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:35.516651    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77_ffc56f52-428f-4e01-bf96-9a42df58b065/main/0.log"
Oct 30 01:41:36.402289 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:36.402239    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-794dc9cdb5-f6dgm_d2949671-a478-4bd4-9da8-d02e13ad22f2/authorino/0.log"
Oct 30 01:41:47.011870 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:47.011837    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8cc7fc69d-vbxkf_aed71d7a-a307-474d-8532-87e110b81265/copy-manifests/0.log"
Oct 30 01:41:47.042039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:47.041985    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8cc7fc69d-vbxkf_aed71d7a-a307-474d-8532-87e110b81265/manager/0.log"
Oct 30 01:41:48.170668 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:48.170637    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-595cc648c-xsksf_e9840e72-b47a-4fb6-b260-515f0f35bfcb/maas-api/0.log"
Oct 30 01:41:49.951842 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:49.951812    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-t92zl_f6cc5223-d801-42fd-82d0-f854c7a44be4/postgres/0.log"
Oct 30 01:41:50.996213 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:50.996159    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-794dc9cdb5-f6dgm_d2949671-a478-4bd4-9da8-d02e13ad22f2/authorino/0.log"
Oct 30 01:41:51.436408 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:51.436378    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-lsd2f_869d4394-feed-4576-a544-4dc0b54c1018/manager/0.log"
Oct 30 01:41:51.626502 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:51.626472    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-ltv2b_93246f44-8c47-4f83-a435-0ec11e3d4436/kuadrant-console-plugin/0.log"
Oct 30 01:41:53.289670 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:53.289640    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7597b65cbf-stpb4_dcad1ee6-9300-4dd7-98fd-6a05d3a6b867/kube-auth-proxy/0.log"
Oct 30 01:41:53.832669 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:53.832638    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-4dwdf_f3f3f72a-7fe6-4495-a7e8-61f971757191/istio-proxy/0.log"
Oct 30 01:41:54.589765 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:54.589722    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-nnnk8_a730d256-143d-4ee5-91ed-e099a288a7f5/istio-proxy/0.log"
Oct 30 01:41:55.203350 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:55.203321    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-xw7jh_a5403c3f-cc0a-434b-9f83-080fe7a76bdb/istio-proxy/0.log"
Oct 30 01:41:56.653236 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:56.653204    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-67bf594cd6-5bhqz_c65bb226-5405-477b-9730-3d33e151004c/main/0.log"
Oct 30 01:41:56.839577 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:56.839546    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-sfwrr_2d2fef78-7a92-4353-8c09-33b082f9d7b8/main/0.log"
Oct 30 01:41:57.360327 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:41:57.360300    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-zhq77_ffc56f52-428f-4e01-bf96-9a42df58b065/main/0.log"
Oct 30 01:41:57.909558 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:41:57.909523    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:42:27.910210 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:42:27.910108    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:42:57.909775 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:42:57.909732    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:43:07.700937 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.700877    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dw966/must-gather-mfkwz"]
Oct 30 01:43:07.701564 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.701545    6011 cpu_manager.go:414] "containerMap: removing container" podUID="70d5a52c-2627-4e9c-9229-01a752fc0827" containerName="maas-api"
Oct 30 01:43:07.701614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.701566    6011 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70d5a52c-2627-4e9c-9229-01a752fc0827" containerName="maas-api"
Oct 30 01:43:07.701688 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.701677    6011 memory_manager.go:365] "RemoveStaleState removing state" podUID="70d5a52c-2627-4e9c-9229-01a752fc0827" containerName="maas-api"
Oct 30 01:43:07.705652 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.705631    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:07.735344 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.735313    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dw966\"/\"openshift-service-ca.crt\""
Oct 30 01:43:07.735610 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.735590    6011 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dw966\"/\"default-dockercfg-rnrqh\""
Oct 30 01:43:07.735704 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.735609    6011 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dw966\"/\"kube-root-ca.crt\""
Oct 30 01:43:07.737654 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.737634    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dw966/must-gather-mfkwz"]
Oct 30 01:43:07.757150 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.757114    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5212ef45-5c5c-4fa7-9140-10727e095d07-must-gather-output\") pod \"must-gather-mfkwz\" (UID: \"5212ef45-5c5c-4fa7-9140-10727e095d07\") " pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:07.757329 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.757166    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2r4r\" (UniqueName: \"kubernetes.io/projected/5212ef45-5c5c-4fa7-9140-10727e095d07-kube-api-access-q2r4r\") pod \"must-gather-mfkwz\" (UID: \"5212ef45-5c5c-4fa7-9140-10727e095d07\") " pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:07.858039 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.858000    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5212ef45-5c5c-4fa7-9140-10727e095d07-must-gather-output\") pod \"must-gather-mfkwz\" (UID: \"5212ef45-5c5c-4fa7-9140-10727e095d07\") " pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:07.858290 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.858053    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2r4r\" (UniqueName: \"kubernetes.io/projected/5212ef45-5c5c-4fa7-9140-10727e095d07-kube-api-access-q2r4r\") pod \"must-gather-mfkwz\" (UID: \"5212ef45-5c5c-4fa7-9140-10727e095d07\") " pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:07.858468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.858444    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/5212ef45-5c5c-4fa7-9140-10727e095d07-must-gather-output\") pod \"must-gather-mfkwz\" (UID: \"5212ef45-5c5c-4fa7-9140-10727e095d07\") " pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:07.879649 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:07.879617    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2r4r\" (UniqueName: \"kubernetes.io/projected/5212ef45-5c5c-4fa7-9140-10727e095d07-kube-api-access-q2r4r\") pod \"must-gather-mfkwz\" (UID: \"5212ef45-5c5c-4fa7-9140-10727e095d07\") " pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:08.021076 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:08.020990    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dw966/must-gather-mfkwz"
Oct 30 01:43:08.317148 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:08.317108    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dw966/must-gather-mfkwz"]
Oct 30 01:43:08.318481 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:43:08.318456    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5212ef45_5c5c_4fa7_9140_10727e095d07.slice/crio-8bab2f78935ea943ed31a4b5756e767e7bea5fa6b0a5934ce66a2ffbb2081d94 WatchSource:0}: Error finding container 8bab2f78935ea943ed31a4b5756e767e7bea5fa6b0a5934ce66a2ffbb2081d94: Status 404 returned error can't find the container with id 8bab2f78935ea943ed31a4b5756e767e7bea5fa6b0a5934ce66a2ffbb2081d94
Oct 30 01:43:09.004686 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:09.004638    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dw966/must-gather-mfkwz" event={"ID":"5212ef45-5c5c-4fa7-9140-10727e095d07","Type":"ContainerStarted","Data":"8bab2f78935ea943ed31a4b5756e767e7bea5fa6b0a5934ce66a2ffbb2081d94"}
Oct 30 01:43:10.011843 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:10.011805    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dw966/must-gather-mfkwz" event={"ID":"5212ef45-5c5c-4fa7-9140-10727e095d07","Type":"ContainerStarted","Data":"a65cec4335790700e49a2213e9e7a2bf318be96e325f850a398879ff997a3a46"}
Oct 30 01:43:10.012274 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:10.011858    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dw966/must-gather-mfkwz" event={"ID":"5212ef45-5c5c-4fa7-9140-10727e095d07","Type":"ContainerStarted","Data":"fcb1108d295c5dbe1c6d55e2d719d4134b3919062d95d36234281b48152aa1d3"}
Oct 30 01:43:10.063900 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:10.063839    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-dw966/must-gather-mfkwz" podStartSLOduration=2.256908738 podStartE2EDuration="3.063826379s" totalImagesPullingTime="806.917641ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 01:43:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:43:10.063360502 +0000 UTC m=+3832.166114479" watchObservedRunningTime="2026-10-30 01:43:10.063826379 +0000 UTC m=+3832.166580349"
Oct 30 01:43:12.342808 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:12.342773    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x2s8q_c64e9e6d-c2b7-4520-b5d9-e8eb86720bab/global-pull-secret-syncer/0.log"
Oct 30 01:43:13.181506 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:13.180552    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p8fxn_5a3f42a4-9d45-4d56-952c-4dd04bb47261/konnectivity-agent/0.log"
Oct 30 01:43:13.525254 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:13.523780    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-254.ec2.internal_7a3c183736bfb0e3d8475eab25325f35/haproxy/0.log"
Oct 30 01:43:20.347645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:20.347610    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-794dc9cdb5-f6dgm_d2949671-a478-4bd4-9da8-d02e13ad22f2/authorino/0.log"
Oct 30 01:43:20.550450 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:20.549660    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-lsd2f_869d4394-feed-4576-a544-4dc0b54c1018/manager/0.log"
Oct 30 01:43:20.661464 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:20.661378    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-ltv2b_93246f44-8c47-4f83-a435-0ec11e3d4436/kuadrant-console-plugin/0.log"
Oct 30 01:43:22.311506 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.311466    6011 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"]
Oct 30 01:43:22.316032 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.315995    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.332593 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.332560    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"]
Oct 30 01:43:22.386872 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.386840    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_34a9b08b-9aed-4952-abdd-41713e001f35/alertmanager/0.log"
Oct 30 01:43:22.389588 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.389557    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-sys\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.389761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.389627    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-lib-modules\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.389761 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.389686    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-proc\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.389885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.389767    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mntwj\" (UniqueName: \"kubernetes.io/projected/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-kube-api-access-mntwj\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.389885 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.389792    6011 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-podres\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.491646 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491589    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-lib-modules\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.491849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491691    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-proc\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.491849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491768    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-lib-modules\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.491849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491792    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-proc\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.491849 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491777    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mntwj\" (UniqueName: \"kubernetes.io/projected/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-kube-api-access-mntwj\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.492102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491847    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-podres\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.492102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491918    6011 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-sys\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.492102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.491988    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-podres\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.492102 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.492062    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-sys\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.493633 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.493612    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_34a9b08b-9aed-4952-abdd-41713e001f35/config-reloader/0.log"
Oct 30 01:43:22.512727 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.512690    6011 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mntwj\" (UniqueName: \"kubernetes.io/projected/021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc-kube-api-access-mntwj\") pod \"perf-node-gather-daemonset-4wtmz\" (UID: \"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc\") " pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.599412 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.599297    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_34a9b08b-9aed-4952-abdd-41713e001f35/kube-rbac-proxy-web/0.log"
Oct 30 01:43:22.644495 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.644442    6011 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:22.707614 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.707564    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_34a9b08b-9aed-4952-abdd-41713e001f35/kube-rbac-proxy/0.log"
Oct 30 01:43:22.816604 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.816550    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_34a9b08b-9aed-4952-abdd-41713e001f35/kube-rbac-proxy-metric/0.log"
Oct 30 01:43:22.927217 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.927101    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_34a9b08b-9aed-4952-abdd-41713e001f35/prom-label-proxy/0.log"
Oct 30 01:43:22.960618 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:22.960582    6011 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"]
Oct 30 01:43:22.962620 ip-10-0-131-254 kubenswrapper[6011]: W1030 01:43:22.962427    6011 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod021dd77d_95ad_4d8d_b48a_6fbb4ccaf6bc.slice/crio-55b2e620e003f91a24373d4a5d03827e33678e5c9f46347bb2f0b3f59879f1d7 WatchSource:0}: Error finding container 55b2e620e003f91a24373d4a5d03827e33678e5c9f46347bb2f0b3f59879f1d7: Status 404 returned error can't find the container with id 55b2e620e003f91a24373d4a5d03827e33678e5c9f46347bb2f0b3f59879f1d7
Oct 30 01:43:23.044229 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.043336    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_34a9b08b-9aed-4952-abdd-41713e001f35/init-config-reloader/0.log"
Oct 30 01:43:23.165939 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.165897    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz" event={"ID":"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc","Type":"ContainerStarted","Data":"fb65168ee38f209c16c749d7f37699a7ce6e5419818f986f1ba366aa91ad3553"}
Oct 30 01:43:23.166142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.165968    6011 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz" event={"ID":"021dd77d-95ad-4d8d-b48a-6fbb4ccaf6bc","Type":"ContainerStarted","Data":"55b2e620e003f91a24373d4a5d03827e33678e5c9f46347bb2f0b3f59879f1d7"}
Oct 30 01:43:23.166142 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.166003    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:23.208191 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.208066    6011 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz" podStartSLOduration=1.208052533 podStartE2EDuration="1.208052533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 01:43:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 01:43:23.205802797 +0000 UTC m=+3845.308556767" watchObservedRunningTime="2026-10-30 01:43:23.208052533 +0000 UTC m=+3845.310806494"
Oct 30 01:43:23.280556 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.280520    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-zbc6h_162c7603-5367-4d79-8ff6-23b05743cf35/kube-state-metrics/0.log"
Oct 30 01:43:23.379939 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.379911    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-zbc6h_162c7603-5367-4d79-8ff6-23b05743cf35/kube-rbac-proxy-main/0.log"
Oct 30 01:43:23.488972 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.488891    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-zbc6h_162c7603-5367-4d79-8ff6-23b05743cf35/kube-rbac-proxy-self/0.log"
Oct 30 01:43:23.598405 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:23.598368    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5dc88c6bb9-9zp98_01d89188-b530-4373-9ea9-b58a4122c8cd/metrics-server/0.log"
Oct 30 01:43:24.457645 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:24.457617    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pztnw_f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/node-exporter/0.log"
Oct 30 01:43:24.562479 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:24.562456    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pztnw_f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/kube-rbac-proxy/0.log"
Oct 30 01:43:24.667575 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:24.667549    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pztnw_f5fbc6ed-d5e4-4b95-8e25-f0ffdc2861bc/init-textfile/0.log"
Oct 30 01:43:24.777387 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:24.777322    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-84864_47f11449-4b9d-44df-8fed-68f31ded1601/kube-rbac-proxy-main/0.log"
Oct 30 01:43:24.884301 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:24.884266    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-84864_47f11449-4b9d-44df-8fed-68f31ded1601/kube-rbac-proxy-self/0.log"
Oct 30 01:43:24.990059 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:24.990034    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-84864_47f11449-4b9d-44df-8fed-68f31ded1601/openshift-state-metrics/0.log"
Oct 30 01:43:25.825662 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:25.825628    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kn2fs_1f7acedd-ae54-4ad5-bef2-da836c8ef78e/prometheus-operator/0.log"
Oct 30 01:43:25.924985 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:25.924950    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-kn2fs_1f7acedd-ae54-4ad5-bef2-da836c8ef78e/kube-rbac-proxy/0.log"
Oct 30 01:43:26.039460 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.039407    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-5tklp_5e25b748-6ffe-488b-a210-24f1b4d1ce57/prometheus-operator-admission-webhook/0.log"
Oct 30 01:43:26.150488 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.150414    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6959d644bc-xfsth_ae0fed46-f827-4a14-879c-b0c4a4fa7db8/telemeter-client/0.log"
Oct 30 01:43:26.256902 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.256872    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6959d644bc-xfsth_ae0fed46-f827-4a14-879c-b0c4a4fa7db8/reload/0.log"
Oct 30 01:43:26.363188 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.363156    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6959d644bc-xfsth_ae0fed46-f827-4a14-879c-b0c4a4fa7db8/kube-rbac-proxy/0.log"
Oct 30 01:43:26.479468 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.479392    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64dbd87c4d-n7xkv_4d53587f-0770-4a39-a369-dcc257c949f5/thanos-query/0.log"
Oct 30 01:43:26.587293 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.587256    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64dbd87c4d-n7xkv_4d53587f-0770-4a39-a369-dcc257c949f5/kube-rbac-proxy-web/0.log"
Oct 30 01:43:26.694137 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.694107    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64dbd87c4d-n7xkv_4d53587f-0770-4a39-a369-dcc257c949f5/kube-rbac-proxy/0.log"
Oct 30 01:43:26.800558 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.800528    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64dbd87c4d-n7xkv_4d53587f-0770-4a39-a369-dcc257c949f5/prom-label-proxy/0.log"
Oct 30 01:43:26.907352 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:26.907316    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64dbd87c4d-n7xkv_4d53587f-0770-4a39-a369-dcc257c949f5/kube-rbac-proxy-rules/0.log"
Oct 30 01:43:27.014442 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:27.014412    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-64dbd87c4d-n7xkv_4d53587f-0770-4a39-a369-dcc257c949f5/kube-rbac-proxy-metrics/0.log"
Oct 30 01:43:27.911440 ip-10-0-131-254 kubenswrapper[6011]: E1030 01:43:27.911398    6011 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-drhmz" containerName="extractor"
Oct 30 01:43:29.184892 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:29.184850    6011 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dw966/perf-node-gather-daemonset-4wtmz"
Oct 30 01:43:30.214552 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:30.214526    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-d6469dd46-qmpmz_7228a710-717e-476f-beac-43807677b0c1/console/0.log"
Oct 30 01:43:33.784917 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:33.784884    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qlw4m_fb95c197-005f-4deb-b563-107e7e0aff33/dns/0.log"
Oct 30 01:43:33.895026 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:33.894993    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qlw4m_fb95c197-005f-4deb-b563-107e7e0aff33/kube-rbac-proxy/0.log"
Oct 30 01:43:34.326814 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:34.326775    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ql49r_66b60042-598e-49bb-8de0-009bd4c1bd6b/dns-node-resolver/0.log"
Oct 30 01:43:35.761065 ip-10-0-131-254 kubenswrapper[6011]: I1030 01:43:35.761028    6011 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6srk_57d0d345-d190-4a7b-b61c-f2f2da64ffc8/node-ca/0.log"